Tracing¶

Eval AI Library includes a tracing module for monitoring LLM calls in production. Available as a lightweight install with minimal overhead, suitable even for high-load systems.

Lite Installation¶

For production tracing without the full evaluation framework:

pip install "eval-ai-library[lite]"

This installs only pydantic and aiohttp, without heavy ML dependencies.

Core Tracer¶

The tracer singleton captures LLM call metadata — execution time, tokens, cost, input/output data:

from eval_lib.tracing import tracer

# Start a trace
trace = tracer.start_trace("my-pipeline")

# Create a span for an LLM call
span = tracer.start_span("generate-answer", SpanType.LLM_CALL, input_data={"prompt": "..."})
# ... your LLM call ...
tracer.end_span(span, output="The answer is 42")

# Or use context manager
with tracer.trace("retrieval", SpanType.RETRIEVAL) as span:
    results = retrieve_documents(query)

tracer.end_trace()

Decorators¶

Automatic function instrumentation with decorators:

from eval_lib.tracing import trace_llm, trace_tool, trace_step

@trace_llm()
async def call_openai(prompt: str) -> str:
    # Automatically traces input, output, duration, cost
    ...

@trace_tool()
async def search_database(query: str) -> list:
    # Traces tool calls
    ...

@trace_step()
async def process_request(request):
    # Traces agent reasoning steps
    ...

LangChain Integration¶

Automatic tracing for all LLM calls in LangChain chains:

from eval_lib.tracing import EvalLibCallbackHandler, callback_handler

from langchain_openai import ChatOpenAI

# Use the global callback handler
llm = ChatOpenAI(callbacks=[callback_handler])

# Or create a new instance
handler = EvalLibCallbackHandler()
llm = ChatOpenAI(callbacks=[handler])

Span Types¶

Available span types for categorizing traces:

SpanType	Description
`LLM_CALL`	LLM API call
`TOOL_CALL`	Tool/function call
`AGENT_STEP`	Agent reasoning step
`REASONING`	Chain-of-thought
`RETRIEVAL`	Document retrieval
`EVALUATION`	Evaluation step
`CUSTOM`	Custom span type

Configuration¶

Configure the tracing endpoint and enable/disable tracing:

from eval_lib.tracing import TracingConfig

config = TracingConfig(
    endpoint="https://your-tracing-server.com",
    api_key="your-api-key",
    enabled=True,
)

Environment variables:

Variable	Description
`EVAL_LIB_TRACING_ENABLED`	Enable/disable tracing
`EVAL_LIB_TRACING_ENDPOINT`	Trace collection endpoint

Context Management¶

The tracing module uses context variables for managing nested spans:

from eval_lib.tracing.context import get_trace_id, set_trace_id, get_parent_span_id

Data Types¶

Trace data is structured with typed models:

from eval_lib.tracing import TraceSpan, SpanType

TraceSpan represents a single unit of work with timing, input/output, status, and metadata.

Async Sender¶

Asynchronous sender for shipping trace data to your collection endpoint with batching:

from eval_lib.tracing.sender import TraceSender