> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Get human annotation count and optimization eligibility

> Returns the optimization-eligibility snapshot for a metric.

`count` is the number of evaluations that have both an AI score and a human-provided score (annotation). For FULL_PROMPT/PARTIAL_PROMPT metrics this means the AI judged the evaluation and a human later reviewed it. For HUMAN_EVALUATION metrics both fields hold the same human-provided value (for analytics compatibility).

`disagreementCount` is the subset of `count` where the human verdict flips the AI's verdict. Both are binarized at the metric's pass threshold (a score at or above it is class 1, anything below is class 0) and the row counts only when the two classes differ. These are the rows the optimizer learns from.

`minAnnotations` and `minDisagreements` are the api-enforced thresholds (mirrored from the metrics-generator hard thresholds). Optimize is rejected early unless `count >= minAnnotations` AND `disagreementCount >= minDisagreements`.

`plannedSplit` is the split the next optimize would make, from the same count gates and split the run applies: the training and validation row counts, or the refusal text when optimize would answer a 400 (too few annotations, too few disagreements, or too few left to train on after the evaluations earlier attempts kept for validation).

> [!WARNING]
>
> You are reading the docs of Galtea 5.10. [Read this page in the latest version](https://docs.galtea.ai/api-reference/metrics/get-human-annotation-count-and-optimization-eligibility.md).


## OpenAPI

````yaml /v5.10/openapi.json get /metrics/{id}/human-annotation-count
openapi: 3.0.0
info:
  version: 1.0.0
  title: Product Management Service API
  description: API documentation for Product Management Service
  contact:
    name: Galtea AI
servers:
  - url: https://api.galtea.ai
security:
  - bearerAuth: []
tags: []
externalDocs:
  description: Galtea Platform Documentation
  url: https://docs.galtea.ai
paths:
  /metrics/{id}/human-annotation-count:
    get:
      tags:
        - metrics
      summary: Get human annotation count and optimization eligibility
      description: >-
        Returns the optimization-eligibility snapshot for a metric.


        `count` is the number of evaluations that have both an AI score and a
        human-provided score (annotation). For FULL_PROMPT/PARTIAL_PROMPT
        metrics this means the AI judged the evaluation and a human later
        reviewed it. For HUMAN_EVALUATION metrics both fields hold the same
        human-provided value (for analytics compatibility).


        `disagreementCount` is the subset of `count` where the human verdict
        flips the AI's verdict. Both are binarized at the metric's pass
        threshold (a score at or above it is class 1, anything below is class 0)
        and the row counts only when the two classes differ. These are the rows
        the optimizer learns from.


        `minAnnotations` and `minDisagreements` are the api-enforced thresholds
        (mirrored from the metrics-generator hard thresholds). Optimize is
        rejected early unless `count >= minAnnotations` AND `disagreementCount
        >= minDisagreements`.


        `plannedSplit` is the split the next optimize would make, from the same
        count gates and split the run applies: the training and validation row
        counts, or the refusal text when optimize would answer a 400 (too few
        annotations, too few disagreements, or too few left to train on after
        the evaluations earlier attempts kept for validation).
      operationId: getMetricHumanAnnotationCount
      parameters:
        - schema:
            type: string
            description: Metric ID
            example: metric_123
          required: true
          description: Metric ID
          name: id
          in: path
      responses:
        '200':
          description: >-
            Annotation counts, thresholds and the planned split retrieved
            successfully
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/MetricOptimizationEligibility'
        '400':
          description: Bad request
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '401':
          description: Unauthorized
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '403':
          description: Forbidden
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '404':
          description: Metric not found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
      security:
        - bearerAuth: []
components:
  schemas:
    MetricOptimizationEligibility:
      type: object
      properties:
        count:
          type: integer
          description: Evaluations with both an AI score and a human score.
        disagreementCount:
          type: integer
          description: >-
            Subset of `count` where the human verdict flips the AI's (AI and
            human on opposite sides of the metric's pass threshold).
        minAnnotations:
          type: integer
          description: Minimum annotations required to optimize this metric.
        minDisagreements:
          type: integer
          description: Minimum disagreements required to optimize this metric.
        minValidationRows:
          type: integer
          description: From this many annotated evaluations a run keeps a validation set.
          example: 30
        plannedSplit:
          type: object
          nullable: true
          properties:
            status:
              type: string
              enum:
                - SPLIT
                - NOT_USED
                - INSUFFICIENT
              description: >-
                `SPLIT`: some evaluations are kept for validation. `NOT_USED`:
                none is, and every one trains. `INSUFFICIENT`: optimize would be
                refused, by a count gate (too few annotations or disagreements)
                or because the evaluations earlier attempts kept for validation
                leave too few to train on.
            trainingRows:
              type: integer
              nullable: true
              description: >-
                Evaluations the optimizer trains on. Null when `status` is
                `INSUFFICIENT`.
              example: 20
            validationRows:
              type: integer
              nullable: true
              description: >-
                Evaluations kept for validation. 0 when `status` is `NOT_USED`,
                null when it is `INSUFFICIENT`.
              example: 10
            reason:
              type: string
              nullable: true
              description: >-
                The 400 message optimize would answer, from the same count gates
                and split the run applies. Set only when `status` is
                `INSUFFICIENT`.
          required:
            - status
            - trainingRows
            - validationRows
            - reason
          additionalProperties: false
          description: >-
            The split the next optimize would make. Null for a metric that
            cannot be optimized: one with no organization, a source other than
            `PARTIAL_PROMPT`, or a legacy revision.
      required:
        - count
        - disagreementCount
        - minAnnotations
        - minDisagreements
        - minValidationRows
        - plannedSplit
    Error:
      type: object
      properties:
        error:
          type: string
          example: Error type
        message:
          type: string
          example: Error message description
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        API key authorization. Pass your API key in the Authorization header as
        a Bearer token. Both new (`gsk_*`) and legacy (`gsk-`) API keys are
        accepted, e.g. `Authorization: Bearer gsk_...` or `Authorization: Bearer
        gsk-...`.

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.