C.H. Robinson uses AI across logistics work such as order entry, pricing, quoting, booking, and tracking.
2
Routing Anthropic models through Vertex AI gives C.H. Robinson more control over where its data goes.
3
Tagging each AI app or workload lets teams connect ownership, spend, model choice, and efficiency metrics.
Summary
Jason Ward describes how C.H. Robinson uses AI throughout logistics operations, from email classification and order entry to pricing, booking, and shipment tracking. The company mainly uses Azure OpenAI and is testing Vertex AI with Anthropic models because the broker arrangement gives it more control over data. For FinOps, Ward starts with ownership. Every AI app, agentic workflow, or model should have a tag. That makes it possible to report cost by application or model and speak with the people who can change usage. He also uses observability data to find prompt bloat, excessive model verbosity, context starvation, retries, cache hits, and reasoning-token ratios. One alert exposed a code bug that repeatedly retried a request. Ward is candid that cost per order and cost per automated task cover only the easy cases. He is experimenting with cost per thought for broader workloads, although its boundaries and benchmarks are still unsettled.
C.H. Robinson applies AI across the logistics workflow
Jason Ward says C.H. Robinson uses AI for order entry and email classification, then describes broader use across pricing, quoting, booking loads, and tracking loads. He says the company is using AI to help with almost anything in logistics, although not every process is fully automated. Developers also use AI in their normal workloads and GitHub Copilot for code assistance, including reading repositories, finding errors, locating changes, and suggesting fixes. Ward distinguishes this from full vibe coding. The aim is to remove mundane work and help developers move faster.
Vertex AI gives the company a controlled route for Anthropic models
C.H. Robinson mainly uses Azure OpenAI and is testing Google Cloud Platform with Vertex AI. Ward says the company can use Vertex AI as a broker for Anthropic models while keeping its data with the company. That matters because direct use of some external APIs may involve sharing data with other companies. He compares the concern with government environments that might keep models and data inside an on-premises data center. The reason is security and reducing the possibility that sensitive information leaves the company.
AI can search observability data for metrics its operator did not know existed
Ward uses an LLM agent in his observability platform to search for useful metrics across AI traces. He compares this with using AI to hack AI. The agent finds signals that Ward may not have known were available, then helps him compare those signals with cost and build better dashboards. He says this only works because the company already has observability for traces and spans. Without that underlying data, teams miss information about AI value, spend, and how well workloads work.
Model verbosity can expose prompt bloat and retry bugs
One metric Ward found measures how chatty an AI or API call is. It looks at calls per trace and helps identify cases where a workload is using too much context or taking too many turns to reach an answer. In one service, a high prompt-bloat result led Ward to contact the owner. The team found a code error that caused the service to loop and retry. Each retry reused the tokens, so the prompt issue was also a source of unnecessary spend. After the code changed, the service left the alert list.
Too little context can waste tokens through repeated failure
Ward is also working on a context-starvation signal. It looks for requests that fail because the model does not have enough context to support the task. The failed request then enters a retry loop until it succeeds, consuming more tokens along the way. Ward says the metric is still being developed and is not fully accurate. It can nevertheless give developers a useful explanation for why a workload is failing. The fix may be a code change that supplies the missing context.
Resource tags are the starting point for AI FinOps
Ward's first advice to a startup worried about a large AI bill is to tag every relevant resource. The tag can identify an AI application, an agentic workflow, a model, or another workload, and it should show who owns it. After ownership is visible, teams can report cost by application or by model. Ward tracks spend and efficiency by model so developers can see which model fits their workload. He also says the right audience matters because developers and team leads may care about different parts of the report.
Spend alone does not show whether an AI workload is useful
Ward says AI spend is only part of the story because usage can rise while a more efficient model produces more work. A cheaper model can also take longer and cost more over the full process. For GitHub Copilot, he does not yet have direct billing access, so he plans to estimate value by surveying hours saved and calculating cost per hour saved. For logistics applications, he reports cost per order and cost per automated task. Cost per order is easy to calculate, but Ward describes these discrete tasks as low-hanging fruit because much of the company's AI work does not fit that shape.
Agentic and conversational workloads need separate measurements
Ward found that his prompt-bloat metric produced misleading results when it treated an agentic workflow like a single-shot or conversational prompt. The top results included agentic workflows that made several prompts, so the metric interpreted their combined context as bloat. He is trying to separate agentic workloads, which keep running across multiple prompts, from conversational workloads, which involve a single prompt-and-response exchange. He says the distinction should make it easier to tell true bloat from normal multi-step behavior, although he is still working out the tagging details.
Cost per thought is an early attempt to measure broad AI work
For work that cannot be counted as an order or discrete automated task, Ward is experimenting with a metric he calls cost per thought. It divides total cost across a trace that includes retrieval, tokens, and conversation, then estimates the cost of the model's full input-to-output process. He admits that the boundaries of a thought are difficult to define and that he still needs data to validate the method. He also tracks reasoning ratio, which compares reasoning and non-reasoning tokens, along with cache hit rate. Ward says cached prompts can produce substantial savings, while model switching is not yet used in his environment.
"First, I would suggest finding the best way that you can to tag the resource because that way you can track the ownership of who owns it."Jason Ward06:23
Who should watch
You are responsible for an AI bill and need a practical starting point for assigning ownership to workloads.
Your observability data contains traces and spans, but you are unsure which signals can reveal waste or application bugs.
You need to measure AI value for work that cannot be reduced to a simple cost-per-task calculation.
