Arize Phoenix for tracing and evaluating business AI assistants
Arize Phoenix traces and evaluates AI assistants so a business can see what each answer relied on and whether changes help.
Updated 2026-08-04
What Arize Phoenix is
Arize Phoenix is a tool from Arize AI for observing applications built on large language models. It records a trace of each step an application takes, such as a model call, a document search or a call to an outside tool, and shows the inputs and outputs of every step in one view. The same view supports evaluation. A set of saved examples can be scored against chosen measures, so a team can compare one prompt or model version against another. Phoenix is built on OpenTelemetry, the common standard for application telemetry, so traces can come from several frameworks and languages. Teams can self-host Phoenix, which keeps trace data on infrastructure the business controls. Phoenix is aimed at the people who build and maintain AI features, and a business owner rarely opens it directly. The team uses it to see what an assistant did, where an answer went astray and whether a change improved results before customers meet it.
Arize Phoenix in plain language
When a business places a language model behind a chat window, an email reply or an internal search box, the answer comes from several steps working together. Reading the question, searching stored documents, calling a model and sometimes calling an outside service all play a part. The final text rarely shows how it was built, so observability tools make those intermediate steps visible after the fact.
Evaluation answers a different question. Rather than studying one conversation, a team gathers representative questions and scores the outputs against criteria it has chosen, such as whether a claim is supported by the source material or whether a request reached a person when it should have. Scores are only as sound as the criteria and examples behind them, so the examples should come from real requests and the criteria should be written down before results are read.
The two views complement each other. A trace explains one run in detail, while an evaluation shows whether a change lifts or lowers quality across many runs. Self-hosting keeps the trace store on infrastructure the business or its provider controls, which puts retention periods, access rights and deletion in the hands of the team.
What businesses use it for
- Tracing why a customer support assistant gave a particular answer, by reviewing the retrieved documents and model steps behind it.
- Comparing two versions of a prompt or model on the same example questions before switching.
- Reviewing a sample of conversations from a lead intake or booking assistant to spot handoffs that failed.
- Keeping a record of evaluation runs so the owner can see whether an assistant has improved or slipped over time.
- Checking each step of a document question answering tool so staff can confirm answers match the source material.
- Finding slow or costly steps in an assistant that calls several outside services.
Who it suits
- Businesses running a customer-facing AI assistant that need to see why it answers as it does.
- Teams with a developer or technical lead who will own the assistant after handover.
- Organisations that need trace data kept on infrastructure they control.
How MILabs sets it up
We start by agreeing with the owner or team lead what the assistant is for and which conversations matter most to review. Where client or patient information passes through the assistant, we recommend a self-hosted instance. We then connect Phoenix to the assistant's existing code so each model call, retrieval step and tool call is recorded. Systems the business uses, such as a CRM or booking platform, stay where they are, and Phoenix receives only the trace data. Access is set by role, so developers can open full traces while other staff see evaluation summaries. Before handover we write down who reviews traces, who can change prompts and who responds when the assistant behaves unexpectedly. MILabs is based in Sydney, works remotely and runs sessions by video call.
Builds it fits
Industries it fits
Arize Phoenix in practice
In a business running a customer-facing assistant, the tracing tool usually sits beside daily work rather than inside it. Staff keep answering enquiries as before. The technical owner follows failures and flags them, while an operations lead chooses which exchanges deserve a closer look, often from a shared list kept in an ordinary tracker.
A typical change follows a repeatable sequence. Someone proposes a change to a prompt, a retrieval setting or the model. Before anything is edited, a reviewer reads the trace of each failing exchange and writes down a suspected cause. Only then is the candidate tested, and a person reads the differences rather than relying on one average score.
Permissions usually follow job function. Those who maintain the code need the complete record, including the wording a customer used and the passages the assistant retrieved. Everyone else works from summaries in which personal details have been removed, and the business names one person who responds when the assistant behaves unexpectedly.
Over time the team keeps a short log of what it learned. Each entry names the example, the suspected cause, the change made and what the next evaluation run showed. A new developer or contractor can then see why a prompt contains a particular instruction without rebuilding that history from memory.
Frequently asked questions
- What should I check on the vendor's site before using Arize Phoenix?
- Arize Phoenix comes from Arize AI, and its documentation at arize.com/phoenix covers the languages and frameworks it supports. Phoenix is built on OpenTelemetry, so traces can come from several frameworks and languages. Self-hosting keeps trace data on infrastructure the business controls. A business that wants its data to stay close can choose that route, and MILabs will walk through the options in the first conversation.
- How do I know whether Arize Phoenix is paying off?
- Measure it against the work it was meant to reduce. Record how long staff spend investigating unexpected assistant answers, and how often those problems reach a customer, before the tool is in use. Then compare the same measures later. Evaluation scores help too, especially when the example questions reflect real customer requests and a person reviews them. The clearest sign is problems getting fixed faster and with more confidence.
- Which industries use Arize Phoenix?
- Phoenix is a developer tool, so the industries that use it are those building AI assistants for client questions, documents or internal knowledge. Legal services, accounting and bookkeeping firms, insurance brokers, real estate agencies, IT managed service providers and software startups may build such assistants. Recruitment agencies and local councils may also use them for screening and public enquiries. What matters most is how much an assistant's answers need checking.
- What do businesses use Arize Phoenix for?
- Businesses use Phoenix to see inside the assistants they run. Typical uses include tracing why a support assistant gave a particular answer, comparing prompt or model versions on the same examples, and reviewing conversations from intake or enquiry assistants to find handoffs that went wrong. Teams also keep evaluation runs on record so changes can be judged over time. Phoenix earns its keep once an assistant handles real requests.
- Does Arize Phoenix need technical skills to run?
- Yes. Someone on the team needs to be comfortable with the programming language the assistant is written in, because tracing is added to application code. Someone also needs to run and update a hosted service if the business chooses self-hosting. Staff can read traces in the interface without writing code. MILabs handles the setup and documents the steps so an internal person or a remote team can maintain it.
