Skip to content
cairnscale

Cairnscale working example

Which enquiries need a person?

Six invented enquiries test a small decision before anyone relies on it.

Compare the same six requests

A baseline that always chose website matched 2 of six labels. A local language model with a fixed prompt matched 4. It sent a product-page problem and a reporting discrepancy for human review. That adds two reviews an operator could have avoided.

A reviewer owns the ambiguous cases. An invalid response also goes to review. An enquiry is never sent or acted on by this exercise. A production pilot needs a larger, representative set, agreed error limits and evidence from the actual model version.

Read all six inputs, labels, prompt and results (JSON) · Download the evaluation runner

Discuss a similar job

Tell us the outcome, who needs it and one way they would check it.

We work entirely online. A description is enough to start. Keep records, credentials and attachments for the transfer method we agree.

Start an email enquiry

Build a project brief online

Enquiry privacy · hello@cairnscale.uk