Runs Service

The Runs Service in the Galtea SDK allows you to manage runs for your products. A run groups the sessions and evaluations of one launch, so a nightly job, a colleague’s launch and a rerun stay separate. This Service is exposed by the galtea.runs object.

Remember that we will be using the galtea object. More information here.

Quick Example

First, initialize the Galtea SDK:

galtea = Galtea(api_key="YOUR_API_KEY")

Open a run and put work inside it:

with galtea.runs.start(product_id=product_id, custom_id="ci-build-42") as run:
    print(f"Opened run #{run.ordinal} ({run.id})")

    # Every session and evaluation created inside the block joins the run.
    session = galtea.sessions.create(version_id=version_id, is_production=False)
    galtea.traces.create_batch(
        session_id=session.id,
        conversation_turns=[
            {"role": "user", "content": "What is the capital of France?"},
            {"role": "assistant", "content": "The capital of France is Paris."},
        ],
    )
    galtea.evaluations.create(session_id=session.id, metrics=[{"name": "Conversation Relevancy"}])
# Leaving the block closes the run, whether the block finished or raised.

Find the run again by the label you gave it:

run = galtea.runs.get_by_custom_id(product_id=product_id, custom_id="ci-build-42")
print(f"Run {run.id} holds {run.session_count} sessions and {run.evaluation_count} evaluations")

Joining work to a run

These methods accept a run_id: sessions.create, sessions.get_or_create, evaluations.create, evaluations.create_single_turn, evaluations.run and traces.create_and_evaluate. Inside a with galtea.runs.start(...) block you can leave run_id out: the SDK reads the open run from the block. An explicit run_id always wins over the block.

You only close a run you opened yourself. A run that a launch opened ends on its own when its last launch ends, so close() finds it already ended and refuses it.

Service Methods

Run

The record of one evaluation launch

Introduction

SDK

CLI

Concepts

Quick Example

Joining work to a run

Service Methods

Run

​Quick Example

​Joining work to a run

​Service Methods

​Related

Run

Quick Example

Joining work to a run

Service Methods

Related