Setup

Create and calibrate an AI judge in minutes

Creating a judge (<5 mins)

1. Go to Monitoring → Judges → New Judge.
2. Specify the criteria that you want to evaluate from your production traffic.
3. Tweak the prompt of the judge until it matches what you are looking for!

That’s it! Historical and future traces will be scored automatically and shown in the dashboard.

Calibrating your judge

For each evaluated item you have the option to mark it as correct or incorrect. This is automatically stored and used to improve the judge over time.