> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# GitHub Actions

> Learn how to integrate Galtea's evaluation capabilities into your GitHub Actions workflow

## Initial Setup

<Note>
  To initialize the `Galtea` class, you need to provide your API key obtained in the [settings page](https://platform.galtea.ai/settings) of the Galtea platform.
</Note>

Before you can use GitHub Actions with Galtea, you need to perform some credentials and variables configuration:

* **Configure Repository Secrets and Variables**
  * Go to your repository's "Settings" tab
  * Navigate to "Secrets and variables" > "Actions"
  * Add the following:
    * Secret: `GALTEA_API_KEY` - Your Galtea API key
    * Variable: `GALTEA_PRODUCT_ID` - Your Galtea Product ID
  * Click "Add" after each entry

## Dependencies

Create a `requirements.txt` file in your repository and add the dependencies required for your project. At minimum, you'll need galtea library:

```txt requirements.txt theme={"system"}
galtea
python-dotenv
```

## Create your GitHub Action

Create a `.github/workflows/evaluate.yml` file in your repository with the following content:

```yml .github/workflows/evaluate.yml theme={"system"}
name: Galtea Evaluation

on:
  push:
  pull_request:
  workflow_dispatch:

jobs:
  evaluate:
    env:
      GALTEA_API_KEY: ${{ secrets.GALTEA_API_KEY }}
      GALTEA_PRODUCT_ID: ${{ vars.GALTEA_PRODUCT_ID }}
      GALTEA_TEST_NAME: ${{ vars.GALTEA_TEST_NAME }}
      GALTEA_ACCURACY_METRIC_NAME: ${{ vars.GALTEA_ACCURACY_METRIC_NAME }}
      GALTEA_COMPLETENESS_METRIC_NAME: ${{ vars.GALTEA_COMPLETENESS_METRIC_NAME }}
      GITHUB_SHA: ${{ github.sha }}
    runs-on: ubuntu-latest
    steps:
     - name: Checkout
       uses: actions/checkout@v4

     - name: Setup Python
       uses: actions/setup-python@v5
       with:
         python-version: '3.10'

     - name: Install dependencies
       run: |
        pip install -r requirements.txt

     - name: Run Evaluation
       run: |
        python evaluate.py
```

## Create your Test Script

Create a `evaluate.py` file in your repository with the following content:

```python theme={"system"}
from datetime import datetime

from _test_helpers import create_test_product
from galtea import Galtea

galtea = Galtea(api_key="YOUR_API_KEY")

run_identifier: str = datetime.now().strftime("%Y%m%d%H%M%S%f")

# Register product via helper (SDK doesn't expose products.create)
PRODUCT_ID: str = create_test_product(
    galtea,
    name=f"docs-github-actions-product-{run_identifier}",
    description="Product for GitHub Actions integration documentation",
)

# Create a test with test cases
test = galtea.tests.create(
    product_id=PRODUCT_ID,
    name=f"github-actions-test-{run_identifier}",
    type="ACCURACY",
    test_file_path="path/to/accuracy_test.csv",
)

# @start github_actions_workflow
# GitHub Actions workflow environment variables would be:
# GALTEA_API_KEY, GALTEA_PRODUCT_ID, GALTEA_TEST_NAME

version = galtea.versions.create(name=f"v1.X-{run_identifier}", product_id=PRODUCT_ID)

test_cases = galtea.test_cases.list(test_id=test.id)

metrics = [{"name": "Factual Accuracy"}]

# Placeholder for where the model's answer would be generated
# In a real scenario, this would involve calling your model with test_case.input
model_answer = "This is a placeholder model answer."

for test_case in test_cases:
    session = galtea.sessions.create(version_id=version.id, test_case_id=test_case.id)
    galtea.inference_results.create_and_evaluate(
        session_id=session.id,
        output=model_answer,
        metrics=metrics,
    )
# @end github_actions_workflow

print(f"Evaluated {len(test_cases)} test cases against version {version.name}")

# Cleanup
galtea.products.delete(product_id=PRODUCT_ID)
```

> **Success!** 🎉 Your GitHub Actions workflow is now configured to run evaluations with Galtea. Each time you push changes, it will automatically evaluate your product using the latest version of your code.