Pricing¶

Eval AI Library tracks API costs for all LLM calls. Here are the supported model prices.

OpenAI¶

Embedding Models¶

Google Gemini¶

Anthropic Claude¶

Cost Estimation¶

Per-Metric Cost¶

Example: Full RAG Evaluation¶

Evaluating 100 test cases with 4 metrics (Answer Relevancy + Faithfulness + Contextual Relevancy + Contextual Recall):

• LLM calls: 100 × (4 + 3 + 3 + 2) = 1,200 calls
• Estimated cost: ~$0.80 with gpt-4o

Cost Tracking in Results¶

When verbose=True (default), the evaluation summary includes cost information:

======================================================================
                     📋 EVALUATION SUMMARY
======================================================================

Overall Results:
  ✅ Passed: 3 / 3
  ❌ Failed: 0 / 3
  📊 Success Rate: 100.0%

Resource Usage:
  💰 Total Cost: $0.034200
  ⏱️  Total Time: 12.45s
  📈 Avg Time per Test: 4.15s

======================================================================

You can also access cost per metric programmatically:

for _, test_results in results:
    for result in test_results:
        for metric in result.metrics_data:
            print(f"{metric.name}: ${metric.evaluation_cost:.4f}")