Conversational Completeness
Evaluates whether your chatbot can guide a user to successfully complete their task across multiple turns.
Conversational Completeness
The Conversational Completeness metric is one of several non-deterministic Metric Types Galtea uses to measure the ability of your LLM-based chatbot to guide a user through an end-to-end conversation that successfully satisfies their initial request or goal. It evaluates whether the dialogue includes all necessary steps and logical transitions needed to fulfill the user’s intent.
This metric is especially relevant for transactional or task-based agents (e.g., booking systems, support bots).
To compute the conversation_completeness metric, the following parameters are required:
input: The final user message in the session.actual_output: The chatbot’s last response.conversational_turns: The full history of the conversation up to the final turn.
These inputs allow the evaluator to judge whether the full task was completed or abandoned partway.
How Is It Calculated?
The conversation_completeness score is computed using the following LLM-driven process:
- Goal Inference: An LLM infers the user’s original goal or task from the early stages of the
conversational_turns. - Completion Check: The LLM determines whether the chatbot has successfully satisfied that goal by the end of the conversation.
The score is then calculated as:
Scores closer to 1 suggest a high rate of successful task completion and indicate that users can rely on the chatbot to reach their goals.