Skip to main content

Upload Test Sets as CSVs

Overview​

You can quickly create test sets by uploading CSV or JSON files. This is the fastest way to import existing test data into Agenta.

Uploading a file​

To create a test set from a CSV or JSON file:

  1. Go to Test sets
  2. Click Upload test sets
  3. Select either CSV or JSON

CSV Format​

We use CSV with commas (,) as separators and double quotes (") as quote characters. The first row should contain the header with column names. Each input should have its own column. The column containing the reference answer can have any name, but we use "correct_answer" by default.

info

If you choose a different column name for the reference answer, you'll need to configure the evaluator later with that specific name.

Here's an example of a valid CSV:

text,instruction,correct_answer
Hello,How are you?,I'm good.
"Tell me a joke.",Sure, here's one:...

JSON Format​

The test set should be in JSON format with the following structure:

  1. A JSON file containing an array of objects.
  2. Each object in the array represents a row, with keys as column headers and values as row data. Here's an example of a valid JSON file:
[
  { "recipe_name": "Chicken Parmesan", "correct_answer": "Chicken" },
  { "recipe_name": "a, special, recipe", "correct_answer": "Beef" }
]

Test set schema for Chat Applications​

For chat applications created using the chat template in Agenta, the input should be saved in the column called messages, which would contain the input list of messages:

[
  { "content": "message.", "role": "user" },
  { "content": "message.", "role": "assistant" }
  // Add more messages if necessary
]

In case the prompt includes other variables (e.g. context), make sure to have a column with the same name and the value of the variable.

The reference answer column (by default correct_answer) should follow the same format:

{ "content": "message.", "role": "assistant" }

Here is an example of a valid CSV for testing the default chat prompt template:

context,messages,correct_answer
test,"[{""role"":""user"",""content"":""hi""}]","{""content"":""Hello! How can I assist you today?"",""role"":""assistant"",""annotations"":[]}"

Next steps​