> ## Documentation Index
> Fetch the complete documentation index at: https://docs.galtea.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Generate or complete metric configuration with AI

> Two modes, selected by the request body:

- **Spec mode** (provide `specificationIds`): generates 1-3 metric configurations per
  specification, optionally creating the metrics automatically.
- **Completion mode** ("Complete With AI", provide `name` + `description` + `evaluationType`):
  completes a single metric the user is authoring. The name/description/evaluationType/tags
  are the source of truth; only the judge prompt and evaluation params are generated (for
  CLASSIFIER, config[0] is { classifierQuestion, evaluationParams, conditions, refused }
  instead). `specificationIds` may carry at most one id as optional context.

The evaluator model of every generated AI-evaluated metric is the deployment's setting for
generated metrics, not a choice of the generator. It answers 500 when that model is not
selectable in the environment. A CLASSIFIER result carries no model.

See [Metrics](https://docs.galtea.ai/concepts/metric).


> [!WARNING]
>
> You are reading the docs of Galtea 5.17. [Read this page in the latest version](https://docs.galtea.ai/api-reference/metrics/generate-or-complete-metric-configuration-with-ai.md).


## OpenAPI

````yaml /v5.17/openapi.json post /metrics/generate-config
openapi: 3.0.0
info:
  version: 1.0.0
  title: Product Management Service API
  description: API documentation for Product Management Service
  contact:
    name: Galtea AI
servers:
  - url: https://api.galtea.ai
security:
  - bearerAuth: []
tags: []
externalDocs:
  description: Galtea Platform Documentation
  url: https://docs.galtea.ai
paths:
  /metrics/generate-config:
    post:
      tags:
        - metrics
      summary: Generate or complete metric configuration with AI
      description: >
        Two modes, selected by the request body:


        - **Spec mode** (provide `specificationIds`): generates 1-3 metric
        configurations per
          specification, optionally creating the metrics automatically.
        - **Completion mode** ("Complete With AI", provide `name` +
        `description` + `evaluationType`):
          completes a single metric the user is authoring. The name/description/evaluationType/tags
          are the source of truth; only the judge prompt and evaluation params are generated (for
          CLASSIFIER, config[0] is { classifierQuestion, evaluationParams, conditions, refused }
          instead). `specificationIds` may carry at most one id as optional context.

        The evaluator model of every generated AI-evaluated metric is the
        deployment's setting for

        generated metrics, not a choice of the generator. It answers 500 when
        that model is not

        selectable in the environment. A CLASSIFIER result carries no model.


        See [Metrics](https://docs.galtea.ai/concepts/metric).
      operationId: generateMetricConfig
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                specificationIds:
                  type: array
                  example:
                    - spec_123
                    - spec_456
                  items:
                    type: string
                  description: >-
                    Spec mode - IDs of the specifications to generate metrics
                    for. Completion mode - at most one id used as context.
                automaticCreation:
                  type: boolean
                  description: >-
                    Spec mode only - when true, creates metrics from generated
                    config
                  example: false
                maxMetricsPerSpecification:
                  type: integer
                  description: >-
                    Spec mode only - maximum number of metrics to generate per
                    specification
                isOnboarding:
                  type: boolean
                  description: Spec mode only - whether this is an onboarding flow
                  example: false
                name:
                  type: string
                  description: >-
                    Completion mode - the user-authored metric name (source of
                    truth)
                description:
                  type: string
                  description: >-
                    Completion mode - the user-authored metric description (the
                    criteria to evaluate). Required
                evaluationType:
                  type: string
                  enum:
                    - PARTIAL_PROMPT
                    - HUMAN_EVALUATION
                    - CLASSIFIER
                  description: >-
                    Completion mode - PARTIAL_PROMPT (binary LLM judge),
                    HUMAN_EVALUATION (0-100 annotator guidelines) or CLASSIFIER
                    (one Classifier question; 400 where not enabled)
                tags:
                  type: array
                  items:
                    type: string
                  description: >-
                    Completion mode - user-provided tags (source of truth,
                    optional)
                productId:
                  type: string
                  description: Completion mode - optional product id for additional context
      responses:
        '200':
          description: >-
            Metric configuration generated successfully. For CLASSIFIER,
            config[0] is { classifierQuestion, evaluationParams, conditions,
            refused }; classifierQuestion is null when one question cannot cover
            every condition.
          content:
            application/json:
              schema:
                type: object
                description: Generated metric configuration suggestion
        '201':
          description: Metrics created from generated configuration
          content:
            application/json:
              schema:
                type: object
                properties:
                  config:
                    type: array
                    description: The generated metric configurations
                    items:
                      type: object
                  metrics:
                    type: array
                    description: The metrics created from them
                    items:
                      $ref: '#/components/schemas/Metric'
        '400':
          description: Invalid request
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '401':
          description: Unauthorized
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '403':
          description: >-
            The caller may not read the specifications or their product, or,
            with automatic creation, may not update the specifications or its
            user groups do not list metric.create. Checked before the generator
            runs.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '500':
          description: >-
            GENERATED_METRICS_EVALUATOR_MODEL names a model that is not
            selectable in this environment
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
        '502':
          description: >-
            The metrics generator failed, or returned a metric for a
            specification outside the request
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Error'
      security:
        - bearerAuth: []
components:
  schemas:
    Metric:
      type: object
      properties:
        id:
          type: string
          example: metric_123
        metricGroupId:
          type: string
          readOnly: true
          example: metric_123
          description: >-
            Identifier shared by every metric in the same revision family.
            Server-managed — derived from `parentMetricId` on create (or
            generated for roots). Cannot be set by the caller.
        parentMetricId:
          type: string
          nullable: true
          example: metric_122
          description: >-
            Id of the direct parent metric. On create, providing this value
            turns the new metric into a revision: it joins the parent's family
            and becomes its only active revision: the family's current active
            revision becomes legacy, whether it is the parent or not. Omit or
            null to create a root metric in a fresh group. On responses, this is
            the recorded parent edge (null for roots).
        organizationId:
          type: string
          nullable: true
          example: org_123
        userId:
          type: string
          nullable: true
          example: user_123
        name:
          type: string
          example: Accuracy
        evaluationParams:
          type: array
          items:
            type: string
          example:
            - input
            - actual_output
            - expected_output
          description: >-
            Ordered list of trace fields the evaluator needs, written in
            snake_case (e.g. `input`, `actual_output`, `expected_output`,
            `retrieval_context`). Determines which data the evaluation engine
            extracts from each trace. Full list of accepted values:
            https://docs.galtea.ai/concepts/metric/evaluation-parameters
        source:
          type: string
          nullable: true
          enum:
            - SELF_HOSTED
            - FULL_PROMPT
            - PARTIAL_PROMPT
            - HUMAN_EVALUATION
            - GEVAL
            - DEEPEVAL
            - DETERMINISTIC
            - CLASSIFIER
          example: PARTIAL_PROMPT
          description: >-
            Evaluation method for the metric. `FULL_PROMPT` is deprecated for
            creation — `POST /metrics` rejects it with a 400. Use
            `PARTIAL_PROMPT` for new AI Evaluation metrics. The value remains in
            the enum because existing FULL_PROMPT metrics are still returned by
            reads and filters. `CLASSIFIER` is the Classifier judge, accepted
            only where the deployment enables it.
        definition:
          type: string
          nullable: true
          description: >-
            Name of the built-in evaluator the metric runs, for example
            `json_field_match`. Null when the metric names none.
        judgePrompt:
          type: string
          nullable: true
          example: Evaluate the accuracy of the response
        tags:
          type: array
          items:
            type: string
          example:
            - accuracy
            - quality
        description:
          type: string
          nullable: true
          example: Measures the accuracy of responses
        documentationUrl:
          type: string
          nullable: true
          example: https://docs.example.com/metrics/accuracy
        evaluatorModelName:
          type: string
          nullable: true
          example: gpt-4
        areEvalParamsTop:
          type: boolean
          nullable: true
          description: >-
            When true, evaluationParams are injected at the top level of the
            evaluator prompt instead of nested inside the conversation context.
        isBeingOptimized:
          type: boolean
          readOnly: true
          description: Whether the metric is currently being optimized.
        optimizationStatus:
          type: string
          nullable: true
          enum:
            - OPTIMIZING
            - READY_FOR_REVIEW
            - FAILED
            - NO_IMPROVEMENT
            - DECLINED
            - ACTIVATED
          readOnly: true
          example: READY_FOR_REVIEW
          description: >-
            Where an optimization attempt stands. Null for a metric that did not
            come from an optimization. A candidate waiting for review is legacy
            until it is activated.
        judgeGenerationSettings:
          type: object
          nullable: true
          properties:
            temperature:
              type: number
              nullable: true
              minimum: 0
              example: 0.3
            topP:
              type: number
              nullable: true
              minimum: 0
              maximum: 1
              example: 0.9
            maxOutputTokens:
              type: integer
              nullable: true
              minimum: 1
              example: 512
              description: >-
                A new metric needs at least 256, so the judge's structured
                answer fits. A metric stored before that floor keeps its cap.
            reasoningEffort:
              type: string
              nullable: true
              example: low
          additionalProperties: false
          description: >-
            The generation settings this metric's judge runs with. Null runs the
            platform defaults. Immutable: changing one creates a revision.
        passThreshold:
          type: number
          minimum: 0
          maximum: 1
          example: 0.7
          description: >-
            Score from 0 to 1 at or above which an evaluation of this metric
            passes. Immutable: changing it creates a revision.
        applicabilityScope:
          type: string
          enum:
            - NONE
            - DEVELOPMENT
            - PRODUCTION
            - ALL
          example: PRODUCTION
          description: >-
            Which sessions get the applicability check before this metric is
            judged. `NONE` for a metric the platform does not score. A metric
            that scores `traces` is never checked, so it takes `NONE` and
            refuses any other value. A FULL_PROMPT metric whose judge prompt
            reads `traces` but does not list it in evaluationParams is stored as
            checked, yet the evaluator judges it without the check. Immutable:
            changing it creates a revision. The check runs only where the
            deployment enables it.
        applicabilityThreshold:
          type: number
          nullable: true
          minimum: 0
          maximum: 1
          example: 0.7
          description: >-
            The lowest probability that the metric applies to a session at which
            it is judged. Null means the platform default, 0.5. Immutable:
            changing it creates a revision.
        classifierQuestion:
          allOf:
            - $ref: '#/components/schemas/ClassifierQuestion'
            - nullable: true
              description: >-
                The one question of a CLASSIFIER metric; null for every other
                source.
        optimizationValidation:
          type: object
          nullable: true
          properties:
            used:
              type: boolean
              description: >-
                Whether any evaluation was held out for validation. False when
                none was: too few annotated evaluations and no session kept from
                an earlier attempt, or no session was eligible for the hold-out.
            heldOutEvaluationIds:
              type: array
              items:
                type: string
              description: >-
                Evaluations held out of the search. Empty when no validation set
                was used.
              example:
                - evaluation_123
            results:
              type: object
              nullable: true
              properties:
                optimization:
                  type: object
                  properties:
                    rowsScored:
                      type: integer
                      minimum: 0
                      description: Rows both prompts scored. Only these rows are compared.
                      example: 25
                    rowsTotal:
                      type: integer
                      minimum: 0
                      description: Rows in the set.
                      example: 25
                    source:
                      type: object
                      properties:
                        correct:
                          type: integer
                          minimum: 0
                          description: >-
                            Rows where the AI verdict matches the human label. A
                            score at or above the metric's pass threshold is a
                            positive verdict.
                          example: 20
                        alignment:
                          type: number
                          nullable: true
                          description: >-
                            Alignment between the AI scores and the human labels
                            on the compared rows. Null when undefined.
                          example: 0.71
                      required:
                        - correct
                        - alignment
                      additionalProperties: false
                    candidate:
                      type: object
                      properties:
                        correct:
                          type: integer
                          minimum: 0
                          description: >-
                            Rows where the AI verdict matches the human label. A
                            score at or above the metric's pass threshold is a
                            positive verdict.
                          example: 20
                        alignment:
                          type: number
                          nullable: true
                          description: >-
                            Alignment between the AI scores and the human labels
                            on the compared rows. Null when undefined.
                          example: 0.71
                      required:
                        - correct
                        - alignment
                      additionalProperties: false
                  required:
                    - rowsScored
                    - rowsTotal
                    - source
                    - candidate
                  additionalProperties: false
                validation:
                  type: object
                  properties:
                    rowsScored:
                      type: integer
                      minimum: 0
                      description: Rows both prompts scored. Only these rows are compared.
                      example: 25
                    rowsTotal:
                      type: integer
                      minimum: 0
                      description: Rows in the set.
                      example: 25
                    source:
                      type: object
                      properties:
                        correct:
                          type: integer
                          minimum: 0
                          description: >-
                            Rows where the AI verdict matches the human label. A
                            score at or above the metric's pass threshold is a
                            positive verdict.
                          example: 20
                        alignment:
                          type: number
                          nullable: true
                          description: >-
                            Alignment between the AI scores and the human labels
                            on the compared rows. Null when undefined.
                          example: 0.71
                      required:
                        - correct
                        - alignment
                      additionalProperties: false
                    candidate:
                      type: object
                      properties:
                        correct:
                          type: integer
                          minimum: 0
                          description: >-
                            Rows where the AI verdict matches the human label. A
                            score at or above the metric's pass threshold is a
                            positive verdict.
                          example: 20
                        alignment:
                          type: number
                          nullable: true
                          description: >-
                            Alignment between the AI scores and the human labels
                            on the compared rows. Null when undefined.
                          example: 0.71
                      required:
                        - correct
                        - alignment
                      additionalProperties: false
                  required:
                    - rowsScored
                    - rowsTotal
                    - source
                    - candidate
                  additionalProperties: false
              required:
                - optimization
                - validation
              additionalProperties: false
              description: >-
                Both prompts scored on the rows the search used and on the
                held-out rows. Null when the optimizer reported nothing.
            rows:
              type: array
              nullable: true
              items:
                type: object
                properties:
                  evaluationId:
                    type: string
                    example: evaluation_123
                  humanScore:
                    type: number
                    description: The human label.
                    example: 1
                  sourceScore:
                    type: number
                    nullable: true
                    description: >-
                      The source prompt's judge score on this row, or null when
                      it failed. A score at or above the metric's pass threshold
                      is a positive verdict.
                    example: 0
                  candidateScore:
                    type: number
                    nullable: true
                    description: >-
                      The optimized prompt's judge score on this row, or null
                      when it failed. A score at or above the metric's pass
                      threshold is a positive verdict.
                    example: 1
                required:
                  - evaluationId
                  - humanScore
                  - sourceScore
                  - candidateScore
                additionalProperties: false
              description: >-
                One entry per held-out row the optimizer reported; a score is
                null when that prompt failed on the row. Null when the optimizer
                reported nothing.
          required:
            - used
            - heldOutEvaluationIds
            - results
            - rows
          additionalProperties: false
          readOnly: true
          description: >-
            How an optimization attempt was validated: the held-out evaluations
            and, once the optimizer reported, both prompts scored on the search
            rows and on the held-out rows. Set on an optimization attempt only,
            and null for any other metric, for an attempt that predates it, and
            for an attempt copied from another organization.
        optimizationRun:
          type: object
          nullable: true
          properties:
            strategies:
              type: array
              items:
                type: string
                enum:
                  - FIX_FALSE_POSITIVES
                  - FIX_FALSE_NEGATIVES
                  - ADD_RUBRIC
                  - SIMPLIFY
                  - BROADEN
              description: The strategies the run wrote candidates with.
              example:
                - ADD_RUBRIC
                - SIMPLIFY
            maxSteps:
              type: integer
              minimum: 1
              description: >-
                The run's step budget, after the optimizer applied its own
                ceiling.
              example: 6
            stepsUsed:
              type: integer
              minimum: 0
              description: >-
                The search steps the run took. 0 when the source prompt already
                met the target.
              example: 4
          required:
            - strategies
            - maxSteps
            - stepsUsed
          additionalProperties: false
          readOnly: true
          description: >-
            What an optimization attempt ran: the strategies, the step budget
            and the steps used. Null while the attempt runs, and when it was not
            reported: an attempt from before the report existed, one run by an
            older optimizer, a failed run, an attempt copied from another
            organization, or a metric that is not an optimization attempt. Null
            never means a run of 0 steps.
        specificationIds:
          type: array
          items:
            type: string
          description: >-
            Specifications linked to this metric. A MEMBER caller sees only the
            specifications of the products it can read.
          example:
            - spec_123
        userGroupIds:
          type: array
          items:
            type: string
          example:
            - ug_123
        createdAt:
          type: string
          format: date-time
        legacyAt:
          type: string
          nullable: true
          format: date-time
          readOnly: true
          description: >-
            Earliest non-null of this metric's own legacy date and its evaluator
            model's `unselectableAt`, so a model no longer selectable makes
            every metric linked to it report as legacy even though the metric
            row itself is untouched. Present only once that date has passed: a
            scheduled future cutoff reports null, because the metric is still
            active until then. Unlike `disabledAt`, this field never carries a
            future date.
        isActive:
          type: boolean
          readOnly: true
          description: >-
            Whether this is the active revision of its metric: its own legacy
            date is unset, it is not deleted and it is not being optimized.
            Unlike a null `legacyAt`, it ignores the evaluator model, so a
            revision on a model that is no longer selectable is still active.
        disabledAt:
          type: string
          nullable: true
          format: date-time
          readOnly: true
          description: >-
            Earliest non-null of this metric's own disabled date and its
            evaluator model's `disabledAt` — a disabled model makes every metric
            linked to it report as disabled even though the metric row itself is
            untouched.
        excludedFromAnalyticsAt:
          type: string
          nullable: true
          format: date-time
          readOnly: true
          description: >-
            When set, results produced by this metric revision do not feed
            analytics.
        excludedByUserId:
          type: string
          nullable: true
          readOnly: true
          description: User who last excluded this metric revision from analytics.
      required:
        - id
        - metricGroupId
        - parentMetricId
        - organizationId
        - userId
        - name
        - evaluationParams
        - source
        - definition
        - judgePrompt
        - tags
        - description
        - documentationUrl
        - evaluatorModelName
        - areEvalParamsTop
        - isBeingOptimized
        - optimizationStatus
        - judgeGenerationSettings
        - passThreshold
        - applicabilityScope
        - applicabilityThreshold
        - classifierQuestion
        - optimizationValidation
        - optimizationRun
        - specificationIds
        - userGroupIds
        - createdAt
        - legacyAt
        - isActive
        - disabledAt
        - excludedFromAnalyticsAt
        - excludedByUserId
    Error:
      type: object
      properties:
        error:
          type: string
          example: Error type
        message:
          type: string
          example: Error message description
    ClassifierQuestion:
      type: object
      properties:
        type:
          type: string
          enum:
            - yes_no
            - scale
        instructions:
          type: string
        goodAnswer:
          type: string
          enum:
            - 'yes'
            - 'no'
        criteria:
          type: object
          nullable: true
          properties:
            'yes':
              type: string
              nullable: true
            'no':
              type: string
              nullable: true
          required:
            - 'yes'
            - 'no'
        levels:
          type: array
          items:
            type: object
            properties:
              description:
                type: string
              score:
                type: number
            required:
              - description
              - score
        doesNotApply:
          type: string
          nullable: true
      required:
        - type
        - instructions
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        API key authorization. Pass your API key in the Authorization header as
        a Bearer token. Both new (`gsk_*`) and legacy (`gsk-`) API keys are
        accepted, e.g. `Authorization: Bearer gsk_...` or `Authorization: Bearer
        gsk-...`.

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.