Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt

Use this file to discover all available pages before exploring further.

The Evaluation Service in the Galtea SDK allows you to manage evaluations for assessing your products. This Service is exposed by the galtea.evaluations object.
Remember that we will be using the galtea object. More information here.

​
Quick Example

First, initialize the Galtea SDK: 
galtea = Galtea(api_key="YOUR_API_KEY")
Create evaluations of a session (one per metric): 
evaluations = galtea.evaluations.create(
    session_id=session_id,
    metrics=[{"name": "Role Adherence"}, {"name": "Conversation Relevancy"}],
)
At some point later, you will be able to list all evaluations for that session: 
evaluations = galtea.evaluations.list(session_id=session_id)
Display the results:
Evaluations start in PENDING status while the evaluation engine processes them. Use wait_for() to poll until all evaluations have completed before reading results.
print(f"Total evaluations: {len(evaluations)}")
scores = [ev.score for ev in evaluations if ev.score is not None]
if len(scores) > 0:
    average_score = sum(scores) / len(scores)
    print(f"Average Score: {average_score}")
else:
    print("No scores available yet")

print("Detailed Results:")
for evaluation in evaluations:
    metric = galtea.metrics.get(metric_id=evaluation.metric_id)
    print(f"Metric: {metric}")
    print(f"    Score: {evaluation.score}")
    print(f"    Reason: {evaluation.reason}")

​
Service Methods

Evaluation

The assessment of an evaluation using a specific metric’s criteria