Skip to content

Exact Match

The ExactMatchMetric performs an exact string comparison between the actual output and the expected output.

How It Works

  1. Optionally strips leading/trailing whitespace from both strings
  2. Optionally converts both strings to lowercase
  3. Compares the two strings for equality
  4. Returns 1.0 if they match, 0.0 otherwise

Parameters

Parameter Type Default Description
threshold float 1.0 Minimum score to pass
case_sensitive bool True Whether comparison is case-sensitive
strip_whitespace bool True Whether to strip leading/trailing whitespace

Required Fields

Field Required
input No
actual_output Yes
expected_output Yes

Usage

from eval_lib.metrics import ExactMatchMetric
from eval_lib.test_case import EvalTestCase
import asyncio

test_case = EvalTestCase(
    input="What is the capital of France?",
    actual_output="Paris",
    expected_output="Paris"
)

metric = ExactMatchMetric(threshold=1.0, case_sensitive=False)
result = asyncio.run(metric.evaluate(test_case))
print(result.score)  # 1.0

Example Scenarios

Pass (1.0)

EvalTestCase(
    input="Translate 'hello' to Spanish",
    actual_output="hola",
    expected_output="Hola"
)
# case_sensitive=False -> match

Fail (0.0)

EvalTestCase(
    input="Translate 'hello' to Spanish",
    actual_output="Hola, which means hello in Spanish",
    expected_output="Hola"
)
# Strings are not equal