> ## Documentation Index
> Fetch the complete documentation index at: https://ezvals.com/llms.txt
> Use this file to discover all available pages before exploring further.

# The @eval Decorator

> Configure and register evaluation functions

The `@eval` decorator marks functions as evaluations. Here's a complete example:

```python theme={null}
from ezvals import eval, EvalContext

@eval(
    dataset="customer_service",
    labels=["production", "critical"],
    input="I want to cancel my subscription",
    metadata={"category": "cancellation"},
    timeout=10.0,
)
async def test_cancellation_handling(ctx: EvalContext):
    ctx.output = await customer_agent(ctx.input)

    assert "cancel" in ctx.output.lower(), "Should address cancellation"
    assert len(ctx.output) > 50, "Response too short"
```

## Configuration Options

### Dataset

Groups related evaluations together:

```python theme={null}
@eval(dataset="sentiment_analysis")
async def test_positive_sentiment(ctx: EvalContext):
    ...

@eval(dataset="sentiment_analysis")
async def test_negative_sentiment(ctx: EvalContext):
    ...
```

<Note>
  If not specified, dataset defaults to the filename (e.g., `evals.py` → `evals`).
</Note>

### Labels

Tags for filtering:

```python theme={null}
@eval(labels=["production", "fast"])
async def test_quick_response(ctx: EvalContext):
    ...

@eval(labels=["experimental", "slow"])
async def test_complex_reasoning(ctx: EvalContext):
    ...
```

Filter with CLI:

```bash theme={null}
ezvals run evals.py --label production
```

### Pre-populated Fields

Set context fields directly in the decorator:

```python theme={null}
@eval(
    input="What is 2 + 2?",
    reference="4",
    metadata={"difficulty": "easy"}
)
async def test_arithmetic(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    assert ctx.output == ctx.reference
```

### Default Score Key

Specify the key for scores (used when assertions fail or with `store(scores=...)`):

```python theme={null}
@eval(input="Classify this text", default_score_key="accuracy")
async def test_classification(ctx: EvalContext):
    ctx.output = await classifier(ctx.input)
    assert ctx.output in ["positive", "negative", "neutral"]
```

### Timeout

Set a maximum execution time:

```python theme={null}
@eval(input="complex task", timeout=5.0)
async def test_with_timeout(ctx: EvalContext):
    ctx.output = await slow_agent(ctx.input)
```

On timeout, the evaluation fails with an error message.

### Target Hook

Run a function before the evaluation body:

```python theme={null}
def call_agent(ctx: EvalContext):
    ctx.output = my_agent(ctx.input)

@eval(input="What's the weather?", target=call_agent)
async def test_weather(ctx: EvalContext):
    # ctx.output already populated by target
    assert "weather" in ctx.output.lower()
```

This separates agent invocation from assertion logic.

### Evaluators

Post-processing functions that add scores:

```python theme={null}
def check_length(result):
    return {
        "key": "length",
        "passed": len(result.output) > 10,
        "notes": f"Output length: {len(result.output)}"
    }

@eval(input="test", evaluators=[check_length])
async def test_response(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    # check_length runs after, adds "length" score
```

### Input Loader

Load test examples dynamically from external sources (databases, APIs):

```python theme={null}
async def fetch_from_db():
    examples = await db.get_test_cases()
    return [{"input": e.prompt, "reference": e.expected} for e in examples]

@eval(dataset="dynamic", input_loader=fetch_from_db)
async def test_from_database(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    assert ctx.output == ctx.reference
```

Each example from the loader becomes a separate eval run. The loader is called lazily at eval time (not at import time), making it ideal for fetching from LangSmith, databases, or APIs.

**Loader return format:**

* Return a list of dicts with `input`, `reference`, and/or `metadata` keys
* Or return objects with `.input`, `.reference`, `.metadata` attributes

<Note>
  `input_loader` cannot be combined with `input=`, `reference=`, or `cases=`.
</Note>

## Sync and Async

Both sync and async functions work—just use `async def` if your code uses `await`.

## Returning Multiple Results

Return a list of `EvalResult` objects for batch evaluations:

```python theme={null}
from ezvals import eval, EvalResult

@eval(dataset="batch")
def test_batch():
    results = []
    for prompt in ["hello", "hi", "hey"]:
        output = my_agent(prompt)
        results.append(EvalResult(
            input=prompt,
            output=output,
            scores=[{"key": "valid", "passed": True}]
        ))
    return results
```

## All Options Reference

| Option              | Type            | Description                                              |
| ------------------- | --------------- | -------------------------------------------------------- |
| `dataset`           | str             | Group name for the evaluation                            |
| `labels`            | list\[str]      | Tags for filtering                                       |
| `input`             | Any             | Pre-populate ctx.input                                   |
| `reference`         | Any             | Pre-populate ctx.reference                               |
| `metadata`          | dict            | Pre-populate ctx.metadata                                |
| `default_score_key` | str             | Key for auto-added scores                                |
| `timeout`           | float           | Max execution time in seconds                            |
| `target`            | callable        | Pre-hook to run before evaluation                        |
| `evaluators`        | list\[callable] | Post-processing score functions                          |
| `input_loader`      | callable        | Async/sync function returning examples                   |
| `cases`             | list\[dict]     | Case definitions that expand into multiple eval variants |