Inside this article
- Agents do different jobs, and models have different strengths. Choosing a model per agent lets each job use the model that suits it.
- The job tells you what to look for: how many requests, how long the material, how careful the reasoning, how many tool calls. Test candidates on your own work, and read the run records.
- The model should be the part you can change. Permissions, approvals and the record should stay the same whichever model does the work.
"Which AI model should we use?" is usually asked as a company-wide question. It works better as a question about one job at a time. A ticket triage agent and a contract review agent have about as much in common as a dispatcher and a solicitor. You would not hire for both roles from one job description.
In PRESHos, each agent is set to the model that suits its job, with Anthropic, OpenAI, Google and xAI models to choose from. This article explains how to make that choice well. The method applies wherever you are able to choose.
Why choose a model per agent?
Models differ in ways that matter to real work. Some are quick and inexpensive and handle short, well-defined tasks well. Some reason more carefully through long or ambiguous material, and take longer and cost more to do it. Some are stronger at working with tools, or at holding a very large document in view.
A single model for every agent means every job gets the same trade-off. The high-volume agent pays for reasoning it does not need. The careful agent may be held to a model chosen for speed.
Choosing per agent removes that compromise. It also spreads your dependence. If a provider changes its prices, retires a model or has a bad week, you change the agents that are affected and leave the rest alone.
What does the job tell you about the model it needs?
Start with the work. Five questions cover most of it.
How many requests? An agent that runs hundreds of times a day makes cost per run and speed the first concerns.
How long is the material? An agent that reads long contracts or two years of ticket history needs a model that can take in a lot at once and still find the detail.
How careful does the reasoning need to be? Sorting a ticket into one of six queues is a narrow judgement. Deciding whether a statement of work conflicts with a master agreement is a wide one.
How many tool calls? An agent that reads records, searches a wiki and prepares updates needs a model that uses tools reliably, in the right order.
Who is waiting? If a person is waiting in a chat or on a phone call, response time matters more than it does for a job that runs overnight.
The examples in the diagram are illustrations. Your own agents will have their own mix, and that mix is what you are matching.
How do you compare models fairly?
Use your own work as the test. Public rankings measure general ability. They do not measure how a model handles your ticket categories or your quoting rules.
A fair comparison has four parts.
Pick real examples. Take a small set of jobs the agent will actually do, including a few awkward ones. Use past work where you already know what a good result looks like.
Keep everything else the same. Same instructions, same tools, same data. Change only the model.
Judge the result first. Have the person who would normally review the work score each result. A model that is cheaper and wrong is the most expensive option.
Then read the cost. In PRESHos every run is kept with its model, its steps, its tokens and its cost. Put two runs of the same job side by side and you can see what each model took to get there. The runs and cost page shows how that is reported.
Here is an example. A distributor tests two models on a partner-enquiry agent. Both give acceptable answers. The run records show that one reaches the answer in fewer steps and reads less on the way. On a job that runs all day, that difference decides it.
What stays the same when the model changes?
Everything that makes the agent safe to use. This is the most important design point in the whole topic.
If your controls live inside a model vendor's product, changing the model means rebuilding the controls. If they live in the platform around the model, the model becomes a setting.
PRESHos reaches every model through one gateway, under the same permissions and the same record. An agent's roles do not change when its model changes. The actions that need approval still need approval. The run is still recorded with who did it, which tools were used and what changed. The model choice page sets this out in a few lines.
For a security reviewer, this means the review does not start again with every model change. The identity, authorization and audit questions were answered at the platform level, and those answers still hold.
When should you change an agent's model?
Treat the choice as something you revisit. Four signals are worth acting on.
The job changed. An agent that used to summarise tickets now drafts customer replies. The bar for writing quality went up.
The results slipped. Reviewers are rejecting more of the agent's work, or correcting the same kind of mistake.
The cost drifted. Cost per run is rising, and the run records show the model working harder than the job should need.
A better option arrived. New models appear often. Run your test set against a new candidate before you move anything.
Change one agent at a time, and compare before and after on the same examples. A model change is a change to how work gets done, so it deserves the same care as any other change to a process.
Who should make the choice?
The decision has three owners, and each brings something different.
The person accountable for the agent knows the job and what good looks like. They judge quality.
Whoever owns the AI budget reads the cost reports and sets the limits.
IT or security confirms that the same controls apply, which on a platform built for model choice is a short conversation.
The agents page explains how each agent has an owner or a named manager, which gives the first of those roles a name.
A sensible default
If you are starting out, keep it simple. Give each new agent a capable general-purpose model, run it on real work for a few weeks, and read the run records. Move the high-volume, simple jobs to a faster model once you have evidence that quality holds. Move the hard jobs up if reviewers are catching reasoning errors.
The aim is a set of agents where each one uses the model that fits its work, and where changing that model is a routine decision. Your permissions, approvals and record should not notice the difference.
If you want to talk through which of your jobs suit which kind of model, get in touch. Tell us which AI tools your team uses today and we will start from there.
