Portkey AI gateway: routing, logs and guardrails for business
How businesses use the Portkey AI gateway for routing, logs and guardrails, and how MILabs sets it up.
Updated 2026-04-28
What Portkey is
The Portkey AI gateway sits between a business's applications and the model providers they call. Instead of connecting each application directly to one model provider, requests pass through Portkey, which forwards them to the chosen model and keeps a record of what was sent and returned. Its documentation covers routing requests, logging and observability, and controls such as guardrails for checking inputs and outputs. Portkey sits in the technical layer of an AI setup, so it is usually used by whoever maintains the systems rather than by staff who only use the finished tool. It can also be self-hosted, which suits businesses that need requests to stay within their own infrastructure. The point of a gateway is to keep the rest of the setup stable. If the business changes model provider, adds a second provider for resilience, or wants to see what each part of its software is spending, those changes are made in one place. It does not replace the applications a business builds on top of it.
Portkey in plain language
A useful way to picture Portkey is as a switchboard in front of a set of AI models. Each time a piece of software wants an answer from a language model, it sends a request. Without a gateway, every application holds its own connection details for each provider and keeps its own records. That works for one application, but it becomes hard to follow once several tools send requests, because nobody has a single place to see what was asked, what came back and which provider handled it.
A gateway gathers those requests onto one path. The applications send their requests to the gateway, and the gateway forwards each one to the model chosen for that job. Answers travel back the same way. Because the traffic passes through one point, the people who maintain the systems can review behaviour, adjust settings and swap providers without editing every application that depends on them.
The idea matters most to technical staff, who need to trace where a request started and where it went. For a business owner, the useful question is simpler: who is responsible for each AI connection, and how would anyone notice if one of them began behaving differently?
What businesses use it for
- Moving between model providers without rewriting every application that relies on them.
- Keeping a log of AI requests and their costs, so the owner can see where spending goes.
- Adding a backup route so a customer-facing assistant keeps working if one provider is slow or unavailable.
- Applying guardrails that check prompts and answers before they reach staff or customers.
- Managing prompts in one place so wording changes do not have to be made in several apps.
- Giving a development team a single point to review how AI is used across internal tools.
Who it suits
- Teams with more than one application calling a model provider directly.
- Businesses that want spending and request logs in one place.
- Owners who need AI requests to stay inside their own infrastructure.
How MILabs sets it up
MILabs starts by sitting down with the owner or team lead to decide what Portkey needs to do for this business. The plan is chosen with the owner, based on how many applications will send requests and which features are needed. Next, we connect Portkey to the model providers the business already uses and to the applications that will call it, so existing systems keep working with the same inputs. Access is set by role, so developers can change routes and keys while others see logs and reports. Before handover, we agree in writing who looks after the gateway, who reviews the logs and who approves changes to routing or guardrails. The easiest first step is to say hello through the enquiry form.
Builds it fits
Industries it fits
Portkey in practice
In a typical business, the gateway sits behind several AI-powered tools added over time. A customer-facing assistant on the website might call one model, an internal search tool might call another, and a reporting script might call a third. Each may have been built by a different person. The gateway brings those calls under one configuration, so the person responsible for the systems can see every connection and change routing from one place.
Day to day, the work is mostly maintenance and review. Someone reads the request logs to confirm the assistant answers the questions it was built for and the guardrails catch the intended inputs. When a provider changes pricing or an interface, the change is made at the gateway and applications keep sending requests as before. Prompts used across several tools can sit in one place, so a wording change is made once.
Access is usually divided. Developers who maintain the gateway hold the keys and routing rules. Operations staff read reports, and department leads raise requests when a tool needs a new capability. The owner can receive a summary of spending and unusual activity. This division is the part a business most often underestimates, and it is what keeps a multi-tool setup manageable.
Frequently asked questions
- What is Portkey?
- Portkey is an AI gateway. It sits between an application and the model providers that application calls, so requests pass through one service. Its documentation describes routing across providers, logging of requests, guardrails for checking inputs and outputs, and tools for managing prompts. A business uses it to keep AI connections organised and visible as more tools are added.
- Can I self-host Portkey?
- Yes, Portkey can be self-hosted, so a business can run the gateway inside its own cloud or on its own servers. That matters when prompts or responses contain client information that must stay within a defined environment. The owner decides who runs the software, how updates are handled and how client information is protected. MILabs talks through these questions before recommending a hosted or self-hosted setup.
- Which builds work well with Portkey?
- Portkey fits builds where several applications send requests to AI models and the owner wants to see their cost and speed. A customer support assistant, an internal staff knowledge assistant and a private document question answering server are common examples. Enquiry triage and policy compliance assistants also benefit from guardrails and logs. Simple automations that call a model once usually do not need a gateway, and MILabs will say so.
- How do I get my team to use Portkey?
- Most staff never open Portkey directly. They use the assistants and applications built on top of it, so the focus is making those tools clear and useful. Give the team a short description of what the assistant can and cannot do, and name one person who answers questions about it. Developers need access to the gateway itself, which is set by role. MILabs can prepare a short internal guide at handover.
- How does MILabs set up Portkey?
- MILabs begins with the owner or team lead, agreeing the purpose of the gateway and a suitable plan. The existing model providers and applications are then connected, with access set by role. Before handover, the business and MILabs agree who looks after the gateway, who reviews logs and who approves changes to routing or guardrails. The easiest first step is to say hello through the enquiry form.
