> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and Evaluate Trace

> Create a trace and immediately evaluate it against specified metrics in a single call.

> [!WARNING]
>
> You are reading the docs of Galtea 5.17. [Read this page in the latest version](https://docs.galtea.ai/sdk/api/trace/create-and-evaluate.md).

## Returns

Returns a tuple containing:

1. A [Trace](https://docs.galtea.ai/v5.17/concepts/product/version/session/trace.md) object
2. A list of [Evaluation](https://docs.galtea.ai/v5.17/concepts/product/version/session/evaluation.md) objects, one for each metric evaluated

## Usage

This method combines creating a trace with its evaluation in a single convenient call. It's the recommended approach for single-turn evaluations, replacing the deprecated `galtea.evaluations.create_single_turn()` method.

### Basic Example

Evaluate by passing [Specification](https://docs.galtea.ai/v5.17/concepts/product/specification.md) IDs so the API resolves their linked metrics automatically (recommended), or by listing metric names explicitly.

> [tab] Using specification IDs
>
> ```python theme={"system"}
> # Evaluate using metrics linked to specifications (no need to list metrics manually)
> trace, evaluations = galtea.traces.create_and_evaluate(
>     session_id=session_id,
>     input="What is the capital of France?",
>     output="The capital of France is Paris.",
>     specification_ids=[specification.id],
> )
> ```

> [tab] Using metrics
>
> ```python theme={"system"}
> # Create trace and evaluate in a single call
> trace, evaluations = galtea.traces.create_and_evaluate(
>     session_id=session_id,
>     input="What is the capital of France?",
>     output="The capital of France is Paris.",
>     metrics=[{"name": "Factual Accuracy"}, {"name": "Answer Relevancy"}],
> )
> ```

### With Pre-computed Scores

```python theme={"system"}
# With pre-computed scores for self-hosted metrics
trace, evaluations = galtea.traces.create_and_evaluate(
    session_id=session_id,
    output="Model response...",
    metrics=[
        {"name": "Factual Accuracy"},
        {"name": self_hosted_metric.name, "score": 0.95},  # Pre-computed score
    ],
)
```

### With Custom Score Calculation

```python theme={"system"}
# With dynamic score calculation using CustomScoreEvaluationMetric
from galtea import CustomScoreEvaluationMetric


class MyMetric(CustomScoreEvaluationMetric):
    def measure(self, *args, actual_output: str | None = None, **kwargs) -> float:
        # Your custom scoring logic
        return 0.95


custom_metric = MyMetric(name=self_hosted_metric.name)

trace, evaluations = galtea.traces.create_and_evaluate(
    session_id=session_id,
    output="Model response...",
    metrics=[
        {"name": "Factual Accuracy"},
        {"score": custom_metric},  # Dynamic score calculation
    ],
)
```

## Parameters

**session\_id** `string`

The session ID to log the trace to.

> [!NOTE]
>
> Instead of a `session_id`, you can reference an existing session by its own conversation id (see the section below). Provide exactly one of the two.

> [accordion] Log to an existing session by your own conversation ID
>
> If you already created the session via the [sessions API](https://docs.galtea.ai/v5.17/sdk/api/session/service.md) with your own `custom_id`, reference it here with `session_custom_id` plus a `version_id` or `product_id` to log and evaluate a turn in one call. If no matching session exists, the call returns a `400` error — create it first via the sessions API.
>
> **session\_custom\_id** `string`
>
> Your own conversation id, as an alternative to `session_id`. Galtea finds an existing session with this id (per version); the session is not created here. Requires `version_id` or `product_id`.
>
> **version\_id** `string`
>
> The version the session is anchored to when using `session_custom_id`. If both anchors are given, `version_id` wins and `product_id` is ignored.
>
> **product\_id** `string`
>
> Product anchor used when no `version_id` is given with `session_custom_id`: Galtea finds the session with this custom id under the product (across its versions).
>
> > [!NOTE]
> >
> > If no session matches the `session_custom_id` and anchor, the call returns a `400` error. Create the session first via the [sessions API](https://docs.galtea.ai/v5.17/sdk/api/session/service.md).

**output** `string | object`

The generated output/response from the AI model. A dict is stored as sent. Galtea reads each element of its `content` list on its own: a Galtea file is stored without its upload signature and must be an accepted type, with at most 20 files and 20 MB. Every other element, such as a file URL on your own host, is stored as sent. A Galtea file of another organization fails the call with a `400`. A metric scores a Galtea file only when its judge model reads that kind of file. See [Content Parts](https://docs.galtea.ai/v5.17/concepts/product/version/session/content-parts.md).

Required unless `output_file_paths` carries the whole answer. Give one of the two, or the call raises a `ValueError`.

**output\_file\_paths** `list[str]`

Files the turn answers with, for a product whose reply is a document. Each entry is a local path the SDK uploads, or a Galtea file reference from an earlier upload, which is attached without uploading it again. Any other URL raises a `ValueError` before anything is uploaded.

One output accepts at most 20 files, 20 MB in total, of the same types an input accepts. See [Upload Output File](https://docs.galtea.ai/v5.17/sdk/api/trace/upload-output-file.md) for the list and for reusing one upload across turns.

A custom scoring function still receives the text of the output, never the files.

> [!NOTE]
>
> **A metric scores a file only when its judge model reads that kind of file.** Only a [Galtea file](https://docs.galtea.ai/v5.17/concepts/product/version/session/content-parts.md#galtea-files), one you uploaded to Galtea or one that Galtea stored from inline media, is sent to a judge. A file on your own host is stored as sent and is never sent to a judge as a file, so it never skips an evaluation: the judge reads its entry as JSON text, in its place in the turn. The kinds a judge reads depend on its model, and the model picker of the metric form shows them for each model. A judge is sent PNG, JPEG, WebP, GIF and PDF files only. The other accepted types are stored, and a Galtea input file of those types still reaches your endpoint, but no judge reads them. The rule is the same for the input and the output, and a metric that reads `conversation_turns` reads both sides of every turn. A metric that reads `traces` also reads the Galtea files of every span, and a metric that reads `retrieval_context` reads the Galtea files in the output of `RETRIEVER` spans.
>
> **Galtea numbers the files of a session.** Turn files come first, in turn order, input before output. The files of every span come next, in span start order. A file that sits in several places of a session takes one number, at its first place, and the judge receives it once. The judge reads each file in its place in the text as `[File N]`, and the session page and the span page show the same `File N` beside each Galtea file.
>
> Before an evaluation runs, Galtea checks every Galtea file on each side the metric reads. One file the judge would not read skips the evaluation, whatever text sits beside it. A skipped evaluation costs no credits, and its reason names the file with its number, its turn and what refused it:
>
> * The judge model cannot read that kind of file.
> * The model catalog does not say whether the judge model reads it.
> * The metric names no judge model.
> * The metric is DeepEval-based: those judges are sent text only.
> * The metric is deterministic: it compares text without a model.
> * The metric is a [classifier](https://docs.galtea.ai/v5.17/concepts/metric/classifier.md) metric: it is sent text only.
> * The file is a type no judge reads, such as DOCX, CSV or TIFF, or an audio file: a judge reads audio only through its transcript.
>
> When the evaluator then reads a Galtea file to send it, it skips the evaluation if the file no longer exists in storage, is empty, or is larger than the 20 MB a judge can receive. The reason names the file.
>
> Audio input with no transcript is a separate case. A metric that reads the input is skipped when no turn of the session has input text or a Galtea input file, and a turn's audio carries no transcript. Text or a Galtea file on another turn is enough, and the reason, `Evaluators read an audio input through its transcript, and no turn carries one.`, names no file.
>
> Human evaluation and self-hosted metrics are never skipped for a file or for audio. On deployments that do not send files to the judge, the evaluator also skips a metric whose judge would read the file, and the reason names the file.
>
> To have the content scored, pick a metric whose judge model reads that kind of file, or log the text your pipeline extracted as the turn input or output, without the file. See [Troubleshooting skipped evaluations](https://docs.galtea.ai/v5.17/concepts/metric/evaluation-parameters.md#troubleshooting-skipped-evaluations).

**metrics** `list[str | CustomScoreEvaluationMetric | dict]`

A list of metrics to evaluate against. Supports multiple formats:

* Strings: Metric names (e.g., `["accuracy", "relevance"]`)
* CustomScoreEvaluationMetric: Objects with dynamic score calculation. Must be initialized with either 'name' or 'id' parameter.
* MetricInput dicts: Format with optional id, name, and score.
  * If `score` is a `float`: Pre-calculated score (requires 'id' or 'name' in the dict).
  * If `score` is a `CustomScoreEvaluationMetric`: Dynamic score calculation.

Optional when `specification_ids` is provided (in which case metrics are resolved from the specifications).

**specification\_ids** `list[str]`

A list of [Specification](https://docs.galtea.ai/v5.17/concepts/product/specification.md) IDs. When provided, the evaluation uses the metrics linked to these specifications.

Can be combined with `metrics` — the API merges and deduplicates by metric ID.

> [!NOTE]
>
> This parameter allows you to evaluate against specific product specifications without manually listing all their associated metrics. At least one of `metrics` or `specification_ids` must be provided.

**input** `string | object`

The input text/prompt. If not provided, will be inferred from the test case linked to the session. A dict is stored as sent. Galtea reads each element of its `content` list on its own: a Galtea file is stored without its upload signature and must be an accepted type, with at most 20 files and 20 MB. Every other element, such as a file URL on your own host, is stored as sent. A Galtea file of another organization fails the call with a `400`. A metric scores a Galtea file only when its judge model reads that kind of file. See [Content Parts](https://docs.galtea.ai/v5.17/concepts/product/version/session/content-parts.md).

**input\_file\_paths** `list[str]`

Files the turn received, for a product that reads documents. Each entry is a local path the SDK uploads, or a Galtea file reference from an earlier upload, which is attached without uploading it again. Any other URL raises a `ValueError` before anything is uploaded.

On a dataset session with no `input`, the files are added to the test case input. A production session accepts files with no `input`: the input is then the files alone. A production session with neither raises a `ValueError`. With `session_custom_id` and only a `product_id`, also pass `input`.

One input accepts at most 20 files, 20 MB in total, of the same types a test case input accepts. See [Upload Output File](https://docs.galtea.ai/v5.17/sdk/api/trace/upload-output-file.md) for the list.

Input files follow the same rule as `output_file_paths` above.

**retrieval\_context** `string`

Context retrieved by a RAG system, if applicable.

**latency** `float`

Latency in milliseconds from model invocation to response.

**usage\_info** `dict[str, int]`

Token usage information from the model call.
Supported keys: `input_tokens`, `output_tokens`, `cache_read_input_tokens`.

**cost\_info** `dict[str, float]`

Cost breakdown for the model call.
Supported keys: `cost_per_input_token`, `cost_per_output_token`, `cost_per_cache_read_input_token`.

**conversation\_simulator\_version** `string`

Version of Galtea's conversation simulator used to generate the input.

**run\_id** `string`

The [Run](https://docs.galtea.ai/v5.17/concepts/product/run.md) the evaluations join. Defaults to the run of the enclosing `galtea.runs.start()` block, when there is one. An explicit value wins over the block.

## Related

* [Create Evaluation](https://docs.galtea.ai/v5.17/sdk/api/evaluation/create.md) - Evaluate an existing session or trace
* [Create Trace](https://docs.galtea.ai/v5.17/sdk/api/trace/create.md) - Create a trace without immediate evaluation
* [Upload Output File](https://docs.galtea.ai/v5.17/sdk/api/trace/upload-output-file.md) - Upload one file and reuse it across turns


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.