Metrics Selection¶
The Metrics tab lets you choose which evaluation metrics to run against your API responses.
Metric Categories¶
Metrics are organized into five categories, accessible via sub-tabs:
RAG¶
Metrics for evaluating Retrieval-Augmented Generation systems:
| Metric | Description | Required Fields |
|---|---|---|
| AnswerRelevancyMetric | Multi-step relevancy assessment | input, actual_output |
| AnswerPrecisionMetric | Precision vs expected output | input, actual_output, expected_output |
| FaithfulnessMetric | Factuality vs retrieval context | actual_output, retrieval_context |
| ContextualRelevancyMetric | Context relevance to the query | input, retrieval_context |
| ContextualPrecisionMetric | Precision of retrieved context | input, actual_output, expected_output, retrieval_context |
| ContextualRecallMetric | Completeness of context | expected_output, retrieval_context |
| BiasMetric | Bias detection in responses | actual_output |
| ToxicityMetric | Toxicity level detection | actual_output |
| RestrictedRefusalMetric | Proper refusal of harmful requests | actual_output |
| GEval | Custom G-Eval scoring | input, actual_output |
| CustomEvalMetric | Custom verdict-based evaluation | input, actual_output |
Agent¶
Metrics for evaluating AI agents and tool use:
| Metric | Description |
|---|---|
| ToolCorrectnessMetric | Verifies correct tool selection |
| TaskSuccessMetric | Evaluates task completion |
| RoleAdherenceMetric | Checks role compliance |
Security¶
Metrics for safety and security evaluation:
| Metric | Description |
|---|---|
| PromptInjectionMetric | Detects prompt injection attacks |
| JailbreakMetric | Detects jailbreak attempts |
| PIILeakageMetric | Checks for PII in responses |
Deterministic¶
Non-LLM metrics using exact matching and pattern rules.
Vector¶
Vector-based similarity metrics.
How to Select Metrics¶
- Click the checkbox next to each metric you want to enable
- The required fields (shown as grey tags beneath each metric) indicate what data is needed
- Ensure your Column Mapping and Response Mapping provide the required fields
Warning
Selecting metrics that require fields not available in your dataset or API response will result in evaluation errors. Check the required fields tags carefully.