Vector Metrics¶

Vector metrics evaluate AI outputs by comparing embedding representations rather than using LLM-based judgment. They are faster and cheaper than LLM-based metrics since they only require an embedding model.

Available Metrics¶

Key Properties¶

• No LLM required — these metrics use embedding models only
• Deterministic — same inputs always produce the same score
• Fast — a single embedding call per text, no multi-step LLM chains
• Cost-effective — embedding API calls are significantly cheaper than chat completions

Import¶

from eval_lib.metrics.vector_metrics import (
    SemanticSimilarityMetric,
    ReferenceMatchMetric,
)

Embedding Providers¶

Vector metrics support the following embedding providers via the embedding_provider parameter: