Skip to content

Google Gemini

Use Google Gemini models with the google: prefix.

Setup

export GOOGLE_API_KEY="AI..."

Available Models

Model Use Case
gemini-2.5-pro-preview Latest, best quality
gemini-2.5-flash-preview Fast and efficient
gemini-2.0-flash Previous generation flash
gemini-2.0-flash-lite Lightweight, ultra-cheap
gemini-1.5-pro Stable production model
gemini-1.5-flash Fast previous generation

Usage

from eval_lib import AnswerRelevancyMetric, FaithfulnessMetric

metric = AnswerRelevancyMetric(model="google:gemini-2.0-flash", threshold=0.7)
metric = FaithfulnessMetric(model="google:gemini-2.5-pro-preview", threshold=0.7)

Direct API Calls

from eval_lib import chat_complete

response, cost = await chat_complete(
    llm="google:gemini-2.0-flash",
    messages=[{"role": "user", "content": "Hello!"}],
    temperature=0.0
)