OpenAI¶

Name: Eval AI Library
Author: Aleksandr Meshkov

OpenAI is the default provider. Models are accessed without a prefix or with the openai: prefix.

Setup¶

export OPENAI_API_KEY="sk-..."

Available Models¶

Model	Use Case
`gpt-4o`	Best quality, recommended for evaluations
`gpt-4o-mini`	Faster, cheaper alternative
`gpt-4`	Previous generation
`gpt-3.5-turbo`	Fast and cheap, lower quality
`o1`	Reasoning model
`o3-mini`	Small reasoning model

Embedding Models¶

Model	Dimensions
`text-embedding-3-small`	1536
`text-embedding-3-large`	3072

Usage¶

from eval_lib import AnswerRelevancyMetric

# Both are equivalent
metric = AnswerRelevancyMetric(model="gpt-4o", threshold=0.7)
metric = AnswerRelevancyMetric(model="openai:gpt-4o", threshold=0.7)

Direct API Calls¶

from eval_lib import chat_complete, get_embeddings

# Chat
response, cost = await chat_complete(
    llm="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    temperature=0.0
)

# Embeddings
embeddings, cost = await get_embeddings(
    model="openai:text-embedding-3-small",
    texts=["Hello world"]
)