> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation Parameters

> Data fields available to metric evaluators during scoring

> [!WARNING]
>
> You are reading the docs of Galtea 5.15. [Read this page in the latest version](https://docs.galtea.ai/concepts/metric/evaluation-parameters.md).

When creating or configuring a metric, you select which parameters are relevant for your evaluation. These parameters are made available to the evaluator during scoring.

* For **AI Evaluation**, the selected parameters are automatically prepended to your `judge_prompt`.
* For **Human Evaluation**, they determine which data fields are displayed to annotators.
* For **Self-Hosted** metrics, evaluation parameters do not apply.
* For **Classifier** metrics, you do not select parameters. They come from the `{{ name }}` placeholders in the question, and only a [short list of names](https://docs.galtea.ai/v5.15/concepts/metric/classifier.md#allowed-placeholders) is allowed.

> [!WARNING]
>
> Only select parameters that your evaluation criteria actually reference. Including unnecessary parameters can reduce evaluation quality by adding noise to the evaluator's context.

## Parameter Reference

<table>
  <thead>
    <tr>
      <th>Parameter</th>
      <th>Description</th>
      <th>Availability</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>**input**</td>
      <td>The prompt or query sent to the model.</td>
      <td>Accuracy, Security & Safety, and Behavior</td>
    </tr>

    <tr>
      <td>**actual\_output**</td>
      <td>The actual output generated by the model.</td>
      <td>Accuracy, Security & Safety, and Behavior</td>
    </tr>

    <tr>
      <td>**expected\_output**</td>
      <td>The ideal answer for the given input. A [field rule](https://docs.galtea.ai/v5.15/concepts/metric/json-field-match.md#field-rules) in a JSON expected output is checked only by the JSON Field Match metrics. Every other metric reads it as plain JSON text.</td>
      <td>Accuracy and Security & Safety</td>
    </tr>

    <tr>
      <td>**context**</td>
      <td>Additional background information provided to the model alongside the input.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**retrieval\_context**</td>
      <td>The context retrieved by your RAG system before sending the user query to your LLM. Derived from your `RETRIEVER` spans; you can also supply it directly and it is stored as a `RETRIEVER` span, not a directly-readable field.</td>
      <td>Accuracy, Security & Safety, and Behavior</td>
    </tr>

    <tr>
      <td>**traces**</td>
      <td>Execution traces from the agent, including tool calls, LLM invocations, and other internal operations.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**expected\_tools**</td>
      <td>List of tools expected to be used by the agent to accomplish the task.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**tools\_used**</td>
      <td>List of tools actually used by the agent during execution (automatically inferred from traces).</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_description**</td>
      <td>The description of the product.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_capabilities**</td>
      <td>The capabilities of the product.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_inabilities**</td>
      <td>The product's known inabilities or restrictions.</td>
      <td>All metrics</td>
    </tr>

    <tr>
      <td>**product\_security\_boundaries** *(deprecated)*</td>
      <td>The security boundaries of the product. Deprecated — no longer accepted for new metrics. Use `product_capabilities` and `product_inabilities` instead.</td>
      <td>N/A (rejected by API for new metrics)</td>
    </tr>

    <tr>
      <td>**user\_persona**</td>
      <td>Information about the user interacting with the agent.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**goal**</td>
      <td>The user's objective in the conversation.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**scenario**</td>
      <td>The context or situation for the conversation.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**stopping\_criterias**</td>
      <td>List of criteria that define when a conversation should end.</td>
      <td>Behavior tests</td>
    </tr>

    <tr>
      <td>**conversation\_turns**</td>
      <td>All turns in a conversation, including user and assistant messages.</td>
      <td>Behavior tests (Human Evaluation only)</td>
    </tr>
  </tbody>
</table>

## Troubleshooting skipped evaluations

When an evaluation runs against a session that does not have all the data the metric needs, Galtea marks the evaluation as **SKIPPED** instead of producing a misleading score. The evaluation's error field describes which parameters are missing, grouped by where you provide them.

The categorized error message looks like this:

```
Metric "Contextual Relevancy" requires data that is not available.

Missing from product settings:
  • product_description - Fill in the "Description" field in your product settings.

Missing from test case:
  • expected_output - Provide an "Expected Output" value on your test cases.

Missing span data:
  • retrieval_context - "retrieval_context" is derived from RETRIEVER spans. Configure tracing so your product emits retrieval steps — see the tracing setup docs.
  • traces - This metric needs span data. Configure your product to send spans — see the tracing setup docs.

Learn more: https://docs.galtea.ai/concepts/metric/evaluation-parameters#troubleshooting-skipped-evaluations
```

Each section points to a different place to look:

<table>
  <thead>
    <tr>
      <th>Section</th>
      <th>Where to fix it</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>**Cannot be read by the judge**</td>
      <td>The value is there, but the judge would not read it. A file that the metric's judge model does not read skips the evaluation, whatever text sits beside it, and the reason names the file, its turn and what refused it. The Run Evaluation dialog greys those metrics out before the run, with the same reason under the name. The [file rule on the Trace page](https://docs.galtea.ai/v5.15/concepts/product/version/session/trace.md#param-input-files) lists every cause and the fix.</td>
    </tr>

    <tr>
      <td>**Missing from product settings**</td>
      <td>Open the product and fill in the missing field (Description, Capabilities, or Inabilities). Capabilities and Inabilities can also be derived from the product's Specifications.</td>
    </tr>

    <tr>
      <td>**Missing from test case**</td>
      <td>Edit the test case and provide the missing value. Some fields (Goal, User Persona, Scenario, Stopping Criteria) are only populated on SCENARIOS-type test cases. If your test cases use a different type, use a metric that does not require these parameters.</td>
    </tr>

    <tr>
      <td>**Missing from endpoint connection's output mapping**</td>
      <td>Edit the endpoint connection and add the missing key to the output mapping. `actual_output` is extracted from the `output` key. See [Templates & Mapping](https://docs.galtea.ai/v5.15/concepts/product/endpoint-connection-configuration.md) for the JSONPath syntax used in mapping values.</td>
    </tr>

    <tr>
      <td>**Missing trace data**</td>
      <td>The session has no conversation turns yet. Run the metric against a session that contains at least one turn.</td>
    </tr>

    <tr>
      <td>**Missing span data**</td>
      <td>The metric needs span-derived data (`traces`, `tools_used`, or `retrieval_context`). Configure your product to send span data — see the [Tracing Agent Operations](https://docs.galtea.ai/v5.15/sdk/tutorials/tracing-agent-operations.md) tutorial for setup, and the [Span concept page](https://docs.galtea.ai/v5.15/concepts/product/version/session/span.md) for what gets captured. `tools_used` (from `TOOL` spans) and `retrieval_context` (from `RETRIEVER` spans) are automatically extracted from your span data; you do not provide them directly. One turn with the data is enough: a `retrieval_context` metric is skipped only when no turn of the session has a `RETRIEVER` span with content.</td>
    </tr>
  </tbody>
</table>

A skipped evaluation consumes no credits. The skips listed above are decided before the evaluation reaches the evaluator. The evaluator can also skip one. Two examples are a metric that does not apply to the session where the [applicability check](https://docs.galtea.ai/v5.15/concepts/metric.md#applicability-check) is on, and, on deployments that do not send files to the judge, a metric whose judge model would read a [Galtea file](https://docs.galtea.ai/v5.15/concepts/product/version/session/content-parts.md#galtea-files) on its turns. Those skips consume no credits either.

Spans can reach Galtea a few seconds after the turn they belong to. So on a run, Galtea waits up to about 9 seconds for late spans before it decides. It does not wait when every turn that misses a span finished more than about 9 seconds ago, because late spans for those turns would already have arrived. When it skips after a wait, the message ends with a line such as `Waited ~9 s for late spans.` A production monitor applies the same wait rule and the same other rules as a run, including the **Cannot be read by the judge** rule and the single-turn limit of deterministic metrics. A monitor decides a session up to about two minutes after the session closes, so it rarely waits.

Once you have provided the missing data, re-run the evaluation against the same session. A **Cannot be read by the judge** section is the exception: no data is missing, so re-running changes nothing until you pick a different metric or log the turn with the text you want scored, without the file.

## Related

> [card] [Evaluation Types](https://docs.galtea.ai/v5.15/concepts/metric/evaluation-types.md)
>
> Understand AI Evaluation, Human Evaluation, and Self-Hosted scoring.

> [card] [Metrics Overview](https://docs.galtea.ai/v5.15/concepts/metric.md)
>
> Browse all available metrics and create custom ones.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.