Installation

Install Evalytic with pip. Choose the extras you need.

Requirements

Python 3.10+ (tested on 3.10, 3.11, 3.12, 3.13)
pip (or any PEP 517 compatible installer)

Install Variants

Core (recommended)

pip install evalytic

Includes CLI, VLM judge support (Gemini, OpenAI, Anthropic, Ollama), Rich terminal output, HTML/JSON reports, and .env loading. ~5MB download.

Dependency	Purpose
`httpx`	HTTP client for API calls
`click`	CLI framework
`rich`	Terminal output formatting
`python-dotenv`	`.env` file support
`Jinja2`	HTML report templates
`Pillow`	Image processing

With Metrics

pip install "evalytic[metrics]"

Adds CLIP Score, LPIPS, and ArcFace local metrics. Once installed, CLIP (text2img) and LPIPS (img2img) are auto-enabled in evaly bench — no flags needed. ~2GB download (PyTorch + model weights).

Large download: The [metrics] extra installs PyTorch (~800MB) and model weights. Once installed, metrics run automatically. Use --no-metrics to disable.

With Generation

pip install "evalytic[generation]"

Adds the fal-client dependency for image generation via fal.ai. Required for evaly bench to generate images.

Everything

pip install "evalytic[all]"

Installs both [generation] and [metrics] extras.

Virtual Environment

We recommend using a virtual environment:

python3 -m venv .venv
source .venv/bin/activate  # macOS/Linux
pip install evalytic

Verify Installation

evaly --help
evaly bench --list-models  # show available models

CLI Aliases

Two entry points are available:

evaly — primary command name
evalytic — full-name alias (same functionality)

Development Install

For contributing to Evalytic or running tests:

git clone https://github.com/evalytic/evalytic.git
cd evalytic
pip install -e ".[dev]"
pytest -v  # run test suite

Quickstart Configuration