> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and Evaluate Trace

> Create a trace and immediately evaluate it against specified metrics in a single call.

> [!WARNING]
>
> You are reading the docs of Galtea 5.18. [Read this page in the latest version](https://docs.galtea.ai/sdk/api/trace/create-and-evaluate.md).

## Returns

Returns a tuple containing:

1. A [Trace](https://docs.galtea.ai/v5.18/concepts/product/version/session/trace.md) object
2. A list of [Evaluation](https://docs.galtea.ai/v5.18/concepts/product/version/session/evaluation.md) objects, one for each metric evaluated

## Usage

This method combines creating a trace with its evaluation in a single convenient call. It's the recommended approach for single-turn evaluations, replacing the deprecated `galtea.evaluations.create_single_turn()` method.

### Basic Example

Evaluate by passing [Specification](https://docs.galtea.ai/v5.18/concepts/product/specification.md) IDs so the API resolves their linked metrics automatically (recommended), or by listing metric names explicitly.

> [tab] Using specification IDs
>
> ```python theme={"system"}
> # Evaluate using metrics linked to specifications (no need to list metrics manually)
> trace, evaluations = galtea.traces.create_and_evaluate(
>     session_id=session_id,
>     input="What is the capital of France?",
>     output="The capital of France is Paris.",
>     specification_ids=[specification.id],
> )
> ```

> [tab] Using metrics
>
> ```python theme={"system"}
> # Create trace and evaluate in a single call
> trace, evaluations = galtea.traces.create_and_evaluate(
>     session_id=session_id,
>     input="What is the capital of France?",
>     output="The capital of France is Paris.",
>     metrics=[{"name": "Factual Accuracy"}, {"name": "Answer Relevancy"}],
> )
> ```

### With Pre-computed Scores

```python theme={"system"}
# With pre-computed scores for self-hosted metrics
trace, evaluations = galtea.traces.create_and_evaluate(
    session_id=session_id,
    output="Model response...",
    metrics=[
        {"name": "Factual Accuracy"},
        {"name": self_hosted_metric.name, "score": 0.95},  # Pre-computed score
    ],
)
```

### With Custom Score Calculation

```python theme={"system"}
# With dynamic score calculation using CustomScoreEvaluationMetric
from galtea import CustomScoreEvaluationMetric


class MyMetric(CustomScoreEvaluationMetric):
    def measure(self, *args, actual_output: str | None = None, **kwargs) -> float:
        # Your custom scoring logic
        return 0.95


custom_metric = MyMetric(name=self_hosted_metric.name)

trace, evaluations = galtea.traces.create_and_evaluate(
    session_id=session_id,
    output="Model response...",
    metrics=[
        {"name": "Factual Accuracy"},
        {"score": custom_metric},  # Dynamic score calculation
    ],
)
```

## Parameters

**session\_id** `string`

The session ID to log the trace to.

> [!NOTE]
>
> Instead of a `session_id`, you can reference an existing session by its own conversation id (see the section below). Provide exactly one of the two.

> [accordion] Log to an existing session by your own conversation ID
>
> If you already created the session via the [sessions API](https://docs.galtea.ai/v5.18/sdk/api/session/service.md) with your own `custom_id`, reference it here with `session_custom_id` plus a `version_id` or `product_id` to log and evaluate a turn in one call. If no matching session exists, the call returns a `400` error — create it first via the sessions API.
>
> **session\_custom\_id** `string`
>
> Your own conversation id, as an alternative to `session_id`. Galtea finds an existing session with this id (per version); the session is not created here. Requires `version_id` or `product_id`.
>
> **version\_id** `string`
>
> The version the session is anchored to when using `session_custom_id`. If both anchors are given, `version_id` wins and `product_id` is ignored.
>
> **product\_id** `string`
>
> Product anchor used when no `version_id` is given with `session_custom_id`: Galtea finds the session with this custom id under the product (across its versions).
>
> > [!NOTE]
> >
> > If no session matches the `session_custom_id` and anchor, the call returns a `400` error. Create the session first via the [sessions API](https://docs.galtea.ai/v5.18/sdk/api/session/service.md).

**output** `string | object`

The generated output/response from the AI model. A dict is stored as sent. Galtea reads each element of its `content` list on its own: a Galtea file is stored without its upload signature and must be an accepted type, with at most 20 files and 20 MB. Every other element, such as a file URL on your own host, is stored as sent. A Galtea file of another organization fails the call with a `400`. A metric scores a Galtea file only when its judge model reads that kind of file. See [Content Parts](https://docs.galtea.ai/v5.18/concepts/product/version/session/content-parts.md).

Required unless `output_file_paths` carries the whole answer. Give one of the two, or the call raises a `ValueError`.

**output\_file\_paths** `list[str]`

Files the turn answers with, for a product whose reply is a document. Each entry is a local path the SDK uploads, or a Galtea file reference from an earlier upload, which is attached without uploading it again. Any other URL raises a `ValueError` before anything is uploaded.

One output accepts at most 20 files, 20 MB in total, of the same types an input accepts. See [Upload Output File](https://docs.galtea.ai/v5.18/sdk/api/trace/upload-output-file.md) for the list and for reusing one upload across turns.

A custom scoring function still receives the text of the output, never the files.

> [!NOTE]
>
> **A metric scores a file only when its judge model reads that kind of file.** Only a [Galtea file](https://docs.galtea.ai/v5.18/concepts/product/version/session/content-parts.md#galtea-files), one you uploaded to Galtea or one that Galtea stored from inline media, is sent to a judge. A file on your own host is stored as sent and is never sent to a judge as a file, so it never skips an evaluation: the judge reads its entry as JSON text, in its place in the turn. The kinds a judge reads depend on its model, and the model picker of the metric form shows them for each model. A judge is sent PNG, JPEG, WebP, GIF and PDF files as files. Every LLM judge model reads a text file (TXT, CSV, Markdown, JSON, XML or HTML, or a file of another type whose bytes are text) as its text: the evaluator decodes it and sends the text after the metric's prompt, under the file's name and number. A DeepEval-based metric reads no file, a text file included. A `data:` URI inside that text is replaced with `[inline <type> data removed]`. The other accepted types, such as DOCX, XLSX or TIFF, are stored, and a Galtea input file of those types still reaches your endpoint, but no judge reads them. The rule is the same for the input and the output, and a metric that reads `conversation_turns` reads both sides of every turn. A metric that reads `traces` also reads the Galtea files of every span, and a metric that reads `retrieval_context` reads the Galtea files in the output of `RETRIEVER` spans.
>
> **Galtea numbers the files of a session.** Turn files come first, in turn order, input before output. The files of every span come next, in span start order. A file that sits in several places of a session takes one number, at its first place, and the judge receives it once. The judge reads each file in its place in the text as `[File N]`, and the session page and the span page show the same `File N` beside each Galtea file.
>
> Before an evaluation runs, Galtea checks every Galtea file on each side the metric reads. One file the judge would not read skips the evaluation, whatever text sits beside it. A skipped evaluation costs no credits, and its reason names the file with its number, its turn and what refused it:
>
> * The judge model cannot read that kind of file.
> * The model catalog does not say whether the judge model reads it.
> * The metric names no judge model.
> * The metric is DeepEval-based: those judges are sent the turn text only, not a file.
> * The metric is deterministic: it compares text without a model.
> * The metric is a [classifier](https://docs.galtea.ai/v5.18/concepts/metric/classifier.md) metric and the file is not a text file: a classifier gets the text of a text file, and no image or PDF.
> * The file is a type no judge reads, such as DOCX, XLSX or TIFF, or an audio file: a judge reads audio only through its transcript.
>
> When the evaluator then reads a Galtea file to send it, it skips the evaluation if the file no longer exists in storage, is empty, or is larger than the 20 MB a judge can receive, and for a text file if its bytes are not text. The evaluator reads UTF-8 (the default), UTF-16 with a byte order mark, and windows-1252 (also named ISO-8859-1 or Latin-1). A file that declares another charset, such as ISO-8859-2, skips although its bytes are text in that charset. The reason names the file. For an LLM judge, Galtea adds no other size limit for text: the file text counts toward the judge model's context window like the rest of the session. When the session does not fit, the model refuses the call and the evaluation fails, as for a long conversation. A classifier metric has its own size limit: see its [limits](https://docs.galtea.ai/v5.18/concepts/metric/classifier.md#limits).
>
> On deployments that send files to the judge, the evaluator also skips an evaluation before it reads any file when the files, on every side the metric reads, total more than one evaluator can hold at once. The reason gives the total and that limit. An evaluation with large files can wait for memory on a busy evaluator before it runs. The wait lasts up to about 25 minutes. After that the evaluation fails as a platform error, and you can retry it.
>
> Audio input with no transcript is a separate case. A metric that reads the input is skipped when no turn of the session has input text or a Galtea input file, and a turn's audio carries no transcript. Text or a Galtea file on another turn is enough, and the reason, `Evaluators read an audio input through its transcript, and no turn carries one.`, names no file.
>
> Human evaluation and self-hosted metrics are never skipped for a file or for audio. On deployments that do not send files to the judge, the evaluator also skips a metric whose judge would read the file, and the reason names the file.
>
> To have the content scored, pick a metric whose judge model reads that kind of file, or log the text your pipeline extracted as the turn input or output, without the file. See [Troubleshooting skipped evaluations](https://docs.galtea.ai/v5.18/concepts/metric/evaluation-parameters.md#troubleshooting-skipped-evaluations).

**metrics** `list[str | CustomScoreEvaluationMetric | dict]`

A list of metrics to evaluate against. Supports multiple formats:

* Strings: Metric names (e.g., `["accuracy", "relevance"]`)
* CustomScoreEvaluationMetric: Objects with dynamic score calculation. Must be initialized with either 'name' or 'id' parameter.
* MetricInput dicts: Format with optional id, name, and score.
  * If `score` is a `float`: Pre-calculated score (requires 'id' or 'name' in the dict).
  * If `score` is a `CustomScoreEvaluationMetric`: Dynamic score calculation.

Optional when `specification_ids` is provided (in which case metrics are resolved from the specifications).

**specification\_ids** `list[str]`

A list of [Specification](https://docs.galtea.ai/v5.18/concepts/product/specification.md) IDs. When provided, the evaluation uses the metrics linked to these specifications.

Can be combined with `metrics` — the API merges and deduplicates by metric ID.

> [!NOTE]
>
> This parameter allows you to evaluate against specific product specifications without manually listing all their associated metrics. At least one of `metrics` or `specification_ids` must be provided.

**input** `string | object`

The input text/prompt. If not provided, will be inferred from the test case linked to the session. A dict is stored as sent. Galtea reads each element of its `content` list on its own: a Galtea file is stored without its upload signature and must be an accepted type, with at most 20 files and 20 MB. Every other element, such as a file URL on your own host, is stored as sent. A Galtea file of another organization fails the call with a `400`. A metric scores a Galtea file only when its judge model reads that kind of file. See [Content Parts](https://docs.galtea.ai/v5.18/concepts/product/version/session/content-parts.md).

**input\_file\_paths** `list[str]`

Files the turn received, for a product that reads documents. Each entry is a local path the SDK uploads, or a Galtea file reference from an earlier upload, which is attached without uploading it again. Any other URL raises a `ValueError` before anything is uploaded.

On a dataset session with no `input`, the files are added to the test case input. A production session accepts files with no `input`: the input is then the files alone. A production session with neither raises a `ValueError`. With `session_custom_id` and only a `product_id`, also pass `input`.

One input accepts at most 20 files, 20 MB in total, of the same types a test case input accepts. See [Upload Output File](https://docs.galtea.ai/v5.18/sdk/api/trace/upload-output-file.md) for the list.

Input files follow the same rule as `output_file_paths` above.

**retrieval\_context** `string`

Context retrieved by a RAG system, if applicable.

**latency** `float`

Latency in milliseconds from model invocation to response.

**usage\_info** `dict[str, int]`

Token usage information from the model call.
Supported keys: `input_tokens`, `output_tokens`, `cache_read_input_tokens`.

**cost\_info** `dict[str, float]`

Cost breakdown for the model call.
Supported keys: `cost_per_input_token`, `cost_per_output_token`, `cost_per_cache_read_input_token`.

**conversation\_simulator\_version** `string`

Version of Galtea's conversation simulator used to generate the input.

**run\_id** `string`

The [Run](https://docs.galtea.ai/v5.18/concepts/product/run.md) the evaluations join. Defaults to the run of the enclosing `galtea.runs.start()` block, when there is one. An explicit value wins over the block.

## Related

* [Create Evaluation](https://docs.galtea.ai/v5.18/sdk/api/evaluation/create.md) - Evaluate an existing session or trace
* [Create Trace](https://docs.galtea.ai/v5.18/sdk/api/trace/create.md) - Create a trace without immediate evaluation
* [Upload Output File](https://docs.galtea.ai/v5.18/sdk/api/trace/upload-output-file.md) - Upload one file and reuse it across turns


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.