Skip to content

Length Check

The LengthCheckMetric validates that the actual output falls within a specified length range, measured in characters or words.

How It Works

  1. Measures the output length using the specified unit (chars or words)
  2. Checks if the length falls within the [min_length, max_length] range
  3. Returns 1.0 if the length is within range, 0.0 otherwise

Parameters

Parameter Type Default Description
threshold float 1.0 Minimum score to pass
min_length int 0 Minimum allowed length (inclusive)
max_length int None Maximum allowed length (inclusive). If None, no upper limit is applied
unit str "chars" Unit of measurement: "chars" or "words"

Required Fields

Field Required
input No
actual_output Yes
expected_output No

Usage

from eval_lib.metrics import LengthCheckMetric
from eval_lib.test_case import EvalTestCase
import asyncio

test_case = EvalTestCase(
    input="Write a brief summary",
    actual_output="The project was completed on time and under budget."
)

metric = LengthCheckMetric(
    threshold=1.0,
    min_length=5,
    max_length=50,
    unit="words"
)
result = asyncio.run(metric.evaluate(test_case))
print(result.score)  # 1.0

Example Scenarios

Pass (1.0)

metric = LengthCheckMetric(min_length=100, max_length=500, unit="chars")
EvalTestCase(
    input="Write a paragraph",
    actual_output="Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."
)
# Character count is within the specified range

Fail (0.0) — too short

metric = LengthCheckMetric(min_length=10, unit="words")
EvalTestCase(
    input="Write a detailed explanation",
    actual_output="It works."
)
# Only 2 words, minimum is 10

Fail (0.0) — too long

metric = LengthCheckMetric(max_length=5, unit="words")
EvalTestCase(
    input="Give a one-word answer",
    actual_output="The answer to your question is yes"
)
# 7 words, maximum is 5