> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Usage

> Learn how to use the SDK in your codebases

Below are some examples of how to use the SDK to interact with the Galtea platform.

### Importing the SDK

```python theme={"system"}
from galtea import Galtea

# Initialize client (replace with your real API key)
galtea = Galtea(api_key="YOUR_API_KEY")
```

<Note>
  To initialize the `Galtea` class, you need to provide your API key obtained in the [settings page](https://platform.galtea.ai/settings) of the Galtea platform.
</Note>

### Registering a Product

Registering a [product](/concepts/product) is the first step to start using the Galtea platform.
Product registration isn't supported via the SDK, so you'll need to do this from the [dashboard](https://platform.galtea.ai/).

### Creating a Version

With a [version](/concepts/product/version), you can track changes to your product over time and compare different implementations.

```python theme={"system"}
# 1) Create a version
version = galtea.versions.create(
    name="v1.0-" + run_identifier,
    product_id=product.id,
    description="Initial version with basic summarization capabilities",
)
```

<Info>
  More information about creating a version can be found [here](/sdk/api/version/service#create-version).
</Info>

### Creating a Test

With a [Test](/concepts/product/test) you define the [test cases](/concepts/product/test/case) to evaluate your [product](/concepts/product).
You can create [accuracy](/concepts/product/test/accuracy-tests) or [security](/concepts/product/test/security-tests) tests depending on your evaluation needs.

```python theme={"system"}
# 2) Create a test
test = galtea.tests.create(
    name="example-test-tutorial-" + run_identifier,
    type="ACCURACY",
    product_id=product.id,
    ground_truth_file_path="path/to/knowledge.md",  # The ground truth is the knowledge base
    language="english",
)
```

<Info>
  More information about creating a test can be found [here](/sdk/api/test/service#create-test).
</Info>

### Creating a Metric

[Metrics](/concepts/metric) help you define the criteria for evaluating the performance of your [product](/concepts/product).
You can create custom [Metrics](/concepts/metric) tailored to your specific use cases.

```python theme={"system"}
# 3) Create a standard metric via API
metric_from_api = galtea.metrics.create(
    name="accuracy-" + run_identifier,
    evaluator_model_name="GPT-4.1",
    source="partial_prompt",
    judge_prompt="Determine whether the actual output is equivalent to the expected output.",
    evaluation_params=["input", "actual_output", "expected_output"],
)


# 4) Create a custom metric entry in the platform, then define local evaluator
metric_from_api_custom = galtea.metrics.create(
    name="keyword-check-" + run_identifier,
    source="self_hosted",
    description="Checks if the 'actual output' contains the keyword 'expected'.",
)

from galtea import CustomScoreEvaluationMetric  # noqa: E402


class MyKeywordMetric(CustomScoreEvaluationMetric):
    def __init__(self) -> None:
        super().__init__(name="keyword-check-" + run_identifier)

    def measure(self, *args, actual_output: str | None = None, **kwargs) -> float:
        """
        Returns 1.0 if 'expected' is in actual_output, else 0.0.
        """
        if actual_output is None:
            return 0.0
```

<Info>
  More information about creating a metric can be found [here](/sdk/api/metric/service#create-metric).
</Info>

### Launching Evaluations

[Evaluations](/concepts/product/version/session/evaluation) assess the performance of a [product](/concepts/product) [version](/concepts/product/version) against [test cases](/concepts/product/test/case) using specific [metrics](/concepts/metric). They are now directly linked to [sessions](/concepts/product/version/session) that contain all inference results.

```python theme={"system"}

keyword_metric = MyKeywordMetric()

# 5) Retrieve test cases for the test
test_cases = galtea.test_cases.list(test_id=test.id)

# 6) Run evaluations for each test case (placeholders used for retriever & product)
for test_case in test_cases:
    # Replace the following with your retrieval and product inference functions
    retrieval_context = None  # your_retriever_function(test_case.input)
    actual_output = "This is the expected output"  # your_product_function(test_case.input, ...)

    session = galtea.sessions.create(version_id=version.id, test_case_id=test_case.id)
    galtea.inference_results.create_and_evaluate(
        session_id=session.id,
        output=actual_output,
        retrieval_context=retrieval_context,
        metrics=[
            {"name": metric_from_api.name},
            {"score": keyword_metric},
        ],
    )
```

<Info>
  More information about launching evaluations can be found [here](/sdk/api/evaluation/service).
</Info>

### Retrieving Evaluation Results

Once the [evaluations](/concepts/product/version/session/evaluation) are completed, you can retrieve them to analyze the results.

```python theme={"system"}

# 7) List sessions for the version and print evaluations
sessions = galtea.sessions.list(version_id=version.id, sort_by_created_at="desc")
for session in sessions:
    evaluations = galtea.evaluations.list(session_id=session.id)
    for evaluation in evaluations:
```

### Working with Pagination

When listing resources that may contain many items (products, tests, sessions, evaluations, etc.), the Galtea SDK uses pagination with a **default limit of 10,000 items** per request.

#### Basic Pagination Example

```python theme={"system"}


# 8) Pagination examples
def fetch_all_test_cases(test_id: str, limit: int = 100) -> list:
    all_test_cases = []
    offset = 0
    while True:
        batch = galtea.test_cases.list(test_id=test_id, offset=offset, limit=limit)
        if not batch:
            break
        all_test_cases.extend(batch)
        if len(batch) < limit:
            break
        offset += limit
```

#### Understanding Pagination Parameters

All list methods in the Galtea SDK accept two pagination parameters:

* **`offset`**: Number of items to skip before starting to collect results (default: `0`)
* **`limit`**: Maximum number of items to return in a single request (default: `10000`)

```python theme={"system"}

# Products/pages examples
first_page_products = galtea.products.list(offset=0, limit=10)
second_page_products = galtea.products.list(offset=10, limit=10)
```

#### Pagination Best Practices

1. **For large datasets**: Use smaller `limit` values (e.g., 100) to reduce memory usage and improve response times
2. **For complete data retrieval**: Implement pagination loops as shown in the example above
3. **For small datasets**: If you know there are fewer than 10,000 items, you can omit pagination parameters

<Warning>
  If you don't specify a `limit` and have more than 10,000 items, only the first 10,000 will be returned. Always implement pagination for complete data retrieval when working with large datasets.
</Warning>

#### Pagination Across Different Resources

The same pagination pattern works for all list operations:

```python theme={"system"}

# Generic pagination examples
products_page = galtea.products.list(offset=0, limit=50)
versions_page = galtea.versions.list(product_id=product.id, offset=0, limit=50)
sessions_page = galtea.sessions.list(version_id=version.id, offset=0, limit=50)
if sessions:
    evaluations_page = galtea.evaluations.list(
        session_id=sessions[0].id,
        offset=0,
        limit=50,
```