Skip to main content
POST
Release a claimed evaluation

Authorizations

Authorization
string
header
required

API key authorization. Pass your API key in the Authorization header as a Bearer token. Both new (gsk_*) and legacy (gsk-) API keys are accepted, e.g. Authorization: Bearer gsk_... or Authorization: Bearer gsk-....

Headers

Prefer
string

Send return=minimal to get no body back from a write and skip reading the resource: a create answers 201 with only Location, and an update answers 204.

Example:

"return=minimal"

Path Parameters

id
string
required

Evaluation ID

Example:

"eval_123"

Response

Evaluation released successfully. The body is the resource as its GET answers it.

id
string
required
Example:

"eval_123"

metricId
string
required
Example:

"metric_123"

sessionId
string
required
Example:

"session_123"

productId
string | null
required

Product the evaluated session belongs to

Example:

"product_123"

userId
string | null
required
Example:

"user_123"

status
enum<string>
required
Available options:
PENDING,
PENDING_HUMAN,
SUCCESS,
FAILED,
SKIPPED,
CANCELLED,
OUTDATED
Example:

"SUCCESS"

inferenceResultId
string | null
required
Example:

"ir_123"

score
number | null
required
Example:

0.95

reason
string | null
required
Example:

"High quality response"

error
string | null
required
canRetry
boolean
required
Example:

false

creditsUsed
integer | null
required
Example:

1

judgeInputTokens
integer | null
required

Tokens the judge received, prompt plus session content

Example:

4100

judgeOutputTokens
integer | null
required

Tokens the judge produced, reasoning included

Example:

16800

judgeReasoningTokens
integer | null
required

Reasoning tokens, a subset of judgeOutputTokens

Example:

12400

conversationSimulatorVersion
string | null
required
Example:

"1.0.0"

humanEvaluatorId
string | null
required

The human evaluator of a HUMAN_EVALUATION metric. On an AI-judged metric, the annotator of the case annotation, or null when the case has none.

humanEvaluatorStartedAt
string<date-time> | null
required

When the human evaluator claimed this HUMAN_EVALUATION evaluation. Always null on an AI-judged metric, because annotating needs no claim.

humanScore
number | null
required

Human-provided score. On a metric a judge scores, it is the annotation of the case (the session, or this turn), which every revision of the metric shows.

humanReason
string | null
required

Human-provided reason, from the same annotation as humanScore on a metric a judge scores.

humanEvaluatorFinishedAt
string<date-time> | null
required

When the human evaluator submitted this HUMAN_EVALUATION evaluation. On an AI-judged metric, when the case annotation was last written.

failedTurns
string[]
required

Conversation turns that failed

monitorId
string | null
required

ID of the monitor that dispatched this evaluation

monitorSettledAtTurn
integer | null
required

Session turn count when the monitor scored it

monitorSamplingPercentage
number | null
required

Sampling percentage in effect when the monitor scored this session

runId
string | null
required

ID of the run that launched this evaluation

deletedAt
string<date-time> | null
required
evaluatedAt
string<date-time> | null
required
metricLegacyAt
string<date-time> | null
required
passed
boolean | null
required

Whether the reported score (the reviewer's when set, else the judge's) reaches the metric's pass threshold. Null when the evaluation has no reported score, is outdated or deleted, is a long classifier input, or its metric is not loaded.

metricPassThreshold
number
required

The pass threshold that decides passed: the metric's, copied onto the evaluation when it was created.

Required range: 0 <= x <= 1
metricDisabledAt
string<date-time> | null
required
metricExcludedFromAnalyticsAt
string<date-time> | null
required
metricExcludedByUserId
string | null
required
testCaseLegacyAt
string<date-time> | null
required
scoreBreakdown
object | null
required

CLASSIFIER only: what the classifier answered; null for every other source.

confidence
number | null
required

CLASSIFIER only: how sure the classifier was, from 0 to 1; null for every other source. Values do not compare across question types.

Required range: 0 <= x <= 1
Example:

0.9

dataRegion
enum<string> | null
required

The data region the evaluation was dispatched with: EU keeps its model calls on EU deployments. Null when unknown: an evaluation dispatched before the region was recorded, or never sent to the evaluator.

Available options:
GLOBAL,
EU,
null
Example:

"EU"

longInput
boolean
required

True for a CLASSIFIER evaluation whose input ran past what the classifier scores reliably (CLASSIFIER_RELIABLE_INPUT_TOKENS judge input tokens) and that no reviewer has scored; false for every other evaluation.

createdAt
string<date-time>
metricRevisionNumber
integer | null

The evaluated revision's number within its metric (R1, R2, R3). Null when the metric is not loaded.

Example:

3

metricRevisionName
string | null

The evaluated revision's own name, or null when it has none or the metric is not loaded.