MILabs
Tools

Opik AI tracing and evaluation for small business

Opik by Comet records and evaluates AI assistants, helping small businesses check answers before and after each change.

Updated 2026-01-13

What Opik is

Opik is an open-source platform from Comet for recording and evaluating applications built on large language models. It keeps a trace of the calls an application makes to a model and the steps around them, so a team can see what an assistant received, what it produced and how long each step took. Developers can also run evaluations against test cases, which gives a measured view of whether a change to a prompt, a model or a workflow has improved the answers. Opik can be self-hosted, which keeps trace data on infrastructure the business controls. It is written for engineers and technical teams. Owners rarely open it themselves, yet they gain when the people running an assistant can explain its behaviour and show evidence after each change. It sits alongside the application it observes. For small businesses, AI tracing and evaluation turns a vague complaint into something measurable, and a short conversation with MILabs can show where that would help most.

Comet: official site

Opik in plain language

Modern AI assistants connect a language model to a few other pieces: the customer's question, a set of instructions, perhaps a document store, and a series of steps that decide what happens next. When something goes wrong, the answer is rarely visible in the final reply alone. A trace is a recorded log of one run through that chain, showing each model call, the text that went in, what came back and how long each step took. Tracing turns a vague complaint such as 'the bot got that wrong' into a specific sequence someone can read.

Evaluation answers a different question. Instead of asking whether one answer looks reasonable, it runs the assistant against a fixed collection of test cases and scores the results, so the same questions can be asked again after every change. Scores may come from a rule, a reference answer or another model acting as a judge. Each method suits different questions, so a sensible team reads the scores alongside the actual answers.

Because the tool observes an assistant rather than making its decisions, the value depends on someone choosing the test cases and reading what the measurements show.

What businesses use it for

  • Reviewing how an assistant answered a customer and why
  • Comparing two versions of a prompt before a change goes live
  • Checking that answers follow approved policy wording
  • Tracking how an assistant's answers shift after a model update
  • Keeping a record of test cases for future changes

Who it suits

  • Teams running a customer-facing assistant who need evidence that its answers are accurate
  • Owners whose developers already build and maintain AI workflows
  • Businesses with client data that must stay on infrastructure they control

How MILabs sets it up

The easiest first step is to say hello through the enquiry form, and we'll take it from there. From there we talk with the owner or team lead about what the assistant should answer and who will check its answers. Together we decide which hosting option suits the business's data needs, including whether a self-hosted install is preferred. The team then connects Opik to the assistant or workflow already running, so traces come from real use. Access is limited to the people who review traces or run evaluations. We finish by agreeing who reviews results, who approves prompt changes and who looks after the setup. MILabs is a Sydney agency and works remotely, with sessions run by video call.

Builds it fits

Industries it fits

Opik in practice

In a business running a customer-facing assistant, the work usually starts with a list of the questions the assistant should handle well. Someone close to the service writes those questions down along with the answers the business would accept. Those examples become the test set. Once the assistant is live, traces collect from real conversations, and a reviewer reads the ones where the assistant hesitated, contradicted itself or gave a figure that did not match the source material.

A typical change cycle looks like this. A developer adjusts a prompt, swaps a model or edits a retrieval step. The test set runs again, the scores are compared with the previous run, and the reviewer checks the answers that moved. Only then does the change go live. The record of runs stays in place, so a question about an earlier prompt can be answered from evidence rather than memory.

With several people touching the same assistant, the arrangement clarifies who may approve a change. The owner decides what good answers look like, the technical contact decides how a change is tested, and a named person signs off on anything touching policy wording. Visible roles matter more than the tool itself, because a trace nobody reads does little for the business.

good questions, straight answers

Frequently asked questions

What is Opik?
Opik is an open-source platform from Comet for recording and evaluating applications built on large language models. It keeps traces of model calls and the steps around them, and it supports evaluations against test cases. Teams can self-host it, which keeps trace data on infrastructure they control. It is an engineering tool, so developers who maintain an AI assistant usually use it, while the business owner reads the results.
How does MILabs set up Opik?
MILabs begins by agreeing with the owner or team lead which hosting option suits the business and who will review results. The team then connects Opik to the assistant or workflow already in use, sets permissions so only the right people see traces, and agrees who looks after the setup afterwards. Each step is tailored to the business, so every engagement is shaped around its own assistant and team.
Where does Opik store data?
Where Opik stores data depends on the hosting option the business chooses. A self-hosted install keeps trace and evaluation data on infrastructure the business controls, while a hosted option keeps it with the vendor. MILabs explains the storage location and retention settings during setup, so the choice is clear, in plain terms, before anything goes live.
When is Opik the wrong choice?
Opik is the wrong choice when a business only needs a simple chatbot that is never measured, or when nobody will review traces or evaluation results. It is also a poor fit for a business with no developer or technical contact to connect it to the application. In those cases, a lighter approach, such as keeping a short reviewed set of sample answers, may serve the business better.
Can my team use Opik without training?
Yes, staff who only use the assistant need no training in Opik because it works behind the application and they never open it. People who review traces or run evaluations should be comfortable with basic technical concepts such as prompts and test cases. During setup, MILabs explains the parts those reviewers will use, so the team knows what to look for.
let's make something happen

Workflow automation: how we can help

Choose how you want to work on this with MILabs. Both options start with an enquiry.

Discovery is free, no obligation from the first chat.

Let us build it for you

We build it and look after it.

Build it yourself, we'll help

You build it with our direction, teaching and a weekly check-in.