Evaluation Service

The Evaluation Service in the Galtea SDK allows you to manage evaluations for assessing your products. This Service is exposed by the galtea.evaluations object.

Remember that we will be using the galtea object. More information here.

Quick Example

First, initialize the Galtea SDK:

galtea = Galtea(api_key="YOUR_API_KEY")

Create evaluations of a session (one per metric):

evaluations = galtea.evaluations.create(
    session_id=session_id,
    metrics=[{"name": "Role Adherence"}, {"name": "Conversation Relevancy"}],
)

At some point later, you will be able to list all evaluations for that session:

evaluations = galtea.evaluations.list(session_id=session_id)

Display the results:

Evaluations start in PENDING status while the evaluation engine processes them. Use wait_for() to poll until all evaluations have completed before reading results.

print(f"Total evaluations: {len(evaluations)}")
scores = [ev.score for ev in evaluations if ev.score is not None]
if len(scores) > 0:
    average_score = sum(scores) / len(scores)
    print(f"Average Score: {average_score}")
else:
    print("No scores available yet")

print("Detailed Results:")
for evaluation in evaluations:
    metric = galtea.metrics.get(metric_id=evaluation.metric_id)
    print(f"Metric: {metric}")
    print(f"    Score: {evaluation.score}")
    print(f"    Reason: {evaluation.reason}")

Service Methods

Evaluation

The assessment of an evaluation using a specific metric’s criteria

Overview Run

Introduction

SDK

CLI

Concepts

Quick Example

Service Methods

Evaluation

​Quick Example

​Service Methods

​Related

Evaluation

Quick Example

Service Methods

Related