> ## Documentation Index
> Fetch the complete documentation index at: https://ezvals.com/llms.txt
> Use this file to discover all available pages before exploring further.

# EvalContext

> The context object that powers evaluation result construction

Every eval function receives an `EvalContext` that accumulates evaluation data and builds into an `EvalResult`.

```python theme={null}
from ezvals import eval, EvalContext

@eval(input="What is 2 + 2?", dataset="demo")
async def my_eval(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    assert ctx.output == "4", f"Expected 4, got {ctx.output}"
```

## Pre-populated Fields

Set context fields directly in the `@eval` decorator:

```python theme={null}
@eval(
    input="What is the capital of France?",
    reference="Paris",
    dataset="geography",
    metadata={"category": "capitals"}
)
async def test_capital(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    assert ctx.output == ctx.reference
```

With `cases=`, `input` and `reference` in each case auto-populate the context fields:

```python theme={null}
@eval(
    dataset="sentiment",
    cases=[
        {"input": "I love this!", "reference": "positive"},
        {"input": "This is terrible", "reference": "negative"},
    ],
)
async def test_sentiment(ctx: EvalContext):
    ctx.output = await analyze_sentiment(ctx.input)
    assert ctx.output == ctx.reference
```

## Setting Fields

### Input, Output, and Reference

```python theme={null}
ctx.input = "Your test input"
ctx.output = "The model's response"
ctx.reference = "Expected output"  # Optional
```

### Metadata

Store additional context for debugging and analysis:

```python theme={null}
ctx.metadata["model"] = "gpt-4"
ctx.metadata["temperature"] = 0.7
```

### Scoring with Assertions

Use assertions to score:

```python theme={null}
ctx.output = await my_agent(ctx.input)

assert ctx.output is not None, "Got no output"
assert "expected" in ctx.output.lower(), "Missing expected content"
```

Failed assertions become failing scores with the message as notes.

### Using store() for Explicit Scoring

For numeric scores or multiple named metrics, use `store()`:

```python theme={null}
# Numeric score (0-1 range)
ctx.store(scores=0.85)

# Multiple scores with different keys
ctx.store(scores=[
    {"passed": True, "key": "format"},
    {"value": 0.9, "key": "relevance"}
])
```

### Setting Multiple Fields at Once

`store()` lets you set multiple context fields in one call:

```python theme={null}
# From agent result
result = await run_agent(ctx.input)
ctx.store(**result, scores=True)  # Spread agent result

# Or set everything explicitly
ctx.store(
    input="test",
    output="response",
    latency=0.5,
    scores=True,
    messages=[{"role": "user", "content": "test"}],
    metadata={"model": "gpt-4"}
)
```

## Auto-Return Behavior

You don't need to explicitly return anything—the context automatically builds into an `EvalResult` when the function completes:

```python theme={null}
@eval(input="test", dataset="demo")
async def my_eval(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    assert ctx.output, "Got empty response"
    # No return needed
```

## Exception Safety

If your evaluation throws an exception, partial data is preserved:

```python theme={null}
@eval(input="test input", dataset="demo")
async def test_with_error(ctx: EvalContext):
    ctx.output = await my_agent(ctx.input)
    # If this fails, input and output are still recorded
    assert ctx.output == "expected"
```

The resulting `EvalResult` will have:

* `input` and `output` preserved
* `error` field with the exception message
* A failing score automatically added

## Default Scoring

If no score is added and no assertions fail, EZVals auto-adds a passing score:

```python theme={null}
@eval(input="test", dataset="demo", default_score_key="pass")
async def test_auto_pass(ctx: EvalContext):
    ctx.output = "result"
    # No explicit score - auto-passes with key "pass"
```

## Custom Parameters

For case-based tests with custom data, store it in `ctx.input`:

```python theme={null}
@eval(
    dataset="demo",
    cases=[
        {"input": {"prompt": "I want a refund", "expected_category": "complaint"}},
        {"input": {"prompt": "Thank you!", "expected_category": "praise"}},
    ],
)
async def test_classification(ctx: EvalContext):
    ctx.output = await classify(ctx.input["prompt"])
    assert ctx.output == ctx.input["expected_category"]
```

## Run Metadata (for Observability)

When your eval function runs, the context includes metadata about the current run and eval. This is useful for tagging traces in LangSmith or other observability tools.

```python theme={null}
@eval(dataset="customer_service", labels=["production"])
async def my_eval(ctx: EvalContext):
    # Run-level metadata (same for all evals in a run)
    print(ctx.run_id)        # "1705312200"
    print(ctx.session_name)  # "model-upgrade"
    print(ctx.run_name)      # "baseline"
    print(ctx.eval_path)     # "evals/"

    # Per-eval metadata (from decorator)
    print(ctx.function_name) # "my_eval"
    print(ctx.dataset)       # "customer_service"
    print(ctx.labels)        # ["production"]

    # Tag traces with this metadata
    ctx.output = await my_agent(ctx.input)
```

### Run-Level Metadata

| Property       | Type        | Description                                  |
| -------------- | ----------- | -------------------------------------------- |
| `run_id`       | str \| None | Unique run identifier (timestamp), read-only |
| `session_name` | str \| None | Session name for the run, read-only          |
| `run_name`     | str \| None | Human-readable run name, read-only           |
| `eval_path`    | str \| None | Path to eval file(s) being run, read-only    |

### Per-Eval Metadata

| Property        | Type               | Description                  |
| --------------- | ------------------ | ---------------------------- |
| `function_name` | str \| None        | Name of the eval function    |
| `dataset`       | str \| None        | Dataset from @eval decorator |
| `labels`        | list\[str] \| None | Labels from @eval decorator  |

## API Reference

| Method                                                                                        | Description                     |
| --------------------------------------------------------------------------------------------- | ------------------------------- |
| `store(input, output, reference, latency, scores, messages, trace_url, metadata, trace_data)` | Set any context fields at once  |
| `build()`                                                                                     | Convert to immutable EvalResult |

### store() Parameters

| Parameter    | Type                 | Description                                       |
| ------------ | -------------------- | ------------------------------------------------- |
| `input`      | Any                  | The test input                                    |
| `output`     | Any                  | The system output                                 |
| `reference`  | Any                  | Expected output                                   |
| `latency`    | float                | Execution time in seconds                         |
| `scores`     | bool/float/dict/list | Score(s) to add                                   |
| `messages`   | list                 | Conversation messages (sets trace\_data.messages) |
| `trace_url`  | str                  | External trace link (sets trace\_data.trace\_url) |
| `metadata`   | dict                 | Merges into ctx.metadata                          |
| `trace_data` | dict                 | Merges into ctx.trace\_data                       |

| Property     | Type      | Description                |
| ------------ | --------- | -------------------------- |
| `input`      | Any       | The test input             |
| `output`     | Any       | The system output          |
| `reference`  | Any       | Expected output (optional) |
| `metadata`   | dict      | Custom metadata            |
| `trace_data` | TraceData | Debug/trace data           |
| `latency`    | float     | Execution time             |
| `scores`     | list      | List of Score objects      |