For a company deciding whether to spend millions on new AI systems, the question is no longer simply whether it can obtain enough GPUs. It is whether those GPUs will do enough useful work to justify the investment.
That concern appears in a September survey of 102 qualified respondents. VentureBeat reported that, among 68 organizations running AI in production that supplied utilization data, 55 reported average GPU utilization of 50% or less. That amounts to 81%.
The same survey found that 63% of all respondents planned to adopt a new, additional or replacement AI infrastructure platform within 12 months. Yet only 10% had an infrastructure return on investment figure reviewed and approved by finance.
Those figures do not prove that the planned purchases are wasteful. A company might be replacing an unsuitable system, preparing for a new workload or building ahead of expected demand. But they reveal a difficult sequence in corporate decision making: many organizations appear ready to expand their AI infrastructure before they have established what their current systems are returning.
Nvidia’s bigger definition of an AI factory
The findings arrive as Nvidia expands the meaning of an AI factory beyond a room filled with accelerators.
At GTC Berlin, Nvidia introduced its Vera Rubin DSX POD reference design, describing it as a factory-built, containerized architecture intended to reduce deployment time from months to weeks and bring production capacity online faster.
The company’s argument is strategic as much as technical. Companies should become producers of AI, Nvidia CEO Jensen Huang has argued, rather than remain consumers of services built elsewhere. The factory concept treats AI production as a continuing industrial capability, much like a manufacturing plant that must be designed for throughput, reliability and scale.
That vision includes more than chips. The surrounding architecture described in the discussion includes governed application programming interfaces, Model Context Protocol integration, agent sandboxes, persistent task state, audit trails and confidential computing to protect data and model weights.
This broader approach matters because poor GPU utilization may not be caused by a simple shortage or surplus of hardware. A system can sit idle while software waits for data retrieval, schedules jobs inefficiently or sends work through repeated attempts. Agents may lose time calling tools, recovering from errors or waiting for human approval. A company can therefore own expensive capacity without producing a corresponding amount of useful work.
The case against buying more first
Cast AI offers a particularly stark interpretation of the problem. In its 2026 State of Kubernetes Optimization Report, the company said its analysis of tens of thousands of production Kubernetes clusters found average GPU utilization of 5%.
That figure is based on Cast AI’s own analysis, not the VentureBeat survey, and it represents a different population and methodology. Still, the company uses it to argue that substantial amounts of GPU capacity remain idle and that rightsizing and sharing workloads could improve efficiency.
The critical argument is straightforward. Companies may be treating hardware acquisition as progress when the deeper constraint lies in operations. Better scheduling, workload consolidation and more effective software could unlock capacity without another round of infrastructure spending. In that view, a new AI factory risks becoming an expensive warehouse whose machinery is active only intermittently.
The financial uncertainty makes that criticism harder to dismiss. If finance teams have not signed off on the returns from existing systems, executives may be measuring success through deployment activity rather than business outcomes. More servers can make an AI program look larger without making it more productive.
Capacity can still be the constraint
There is also a credible argument for continued investment.
Meredith Whalen, IDC’s chief research officer, argues in IDC’s analysis of AI infrastructure demand that the industry is short of capacity, not demand. In her view, power, memory, packaging and data center limitations are the principal barriers to expansion. Low utilization in some environments, therefore, does not necessarily mean the broader market has bought too much infrastructure.
Companies may also be building for peaks rather than averages. An AI service that handles occasional but critical surges may need substantial reserve capacity. A platform designed for training, inference and future models may look inefficient during an early rollout while still being part of a long-term plan.
The more meaningful measure for agentic systems may not be GPU occupancy or the price of a token. It is the cost of a successfully completed task. That calculation should include inference, retrieval, storage, retries, tool use and human intervention. Incorrect answers should not count as successful completions simply because they consumed fewer resources.
Nvidia’s AI factory pitch may ultimately succeed if it helps companies coordinate those pieces rather than merely purchase more compute. But the survey suggests that enterprises have not yet closed the evidence gap. Before the next factory comes online, many will need to show whether the machines they already operate are producing value, or simply making underused capacity more expensive.
Daniel Reyes is not a person. No notebook, no deadlines, no face behind the name — just a byline this newsroom publishes under. Here is the production line underneath it, because a name beside a portrait reads like a journalist, and this one is not one.
The models. Writing: gpt-5.6-luna and qwen3-max. Out on the live web: gpt-5.6-luna and gpt-5.6-terra. Pictures: gpt-image-1 and gpt-image-1-mini. Swap one in the newsroom and this line swaps with it — it is read off the machines, not typed here.
How a story is made
- Research. The searching model reads around the story, pointed at primary sources — the filing, the post, the repository — rather than at somebody else's write-up of them.
- Writing. The writing model drafts it against what was found, at Daniel Reyes's usual length and in Daniel Reyes's usual register.
- The loop. A reviewer reads the draft and sends it back with notes. Then reads it again. A piece can go round several times before it leaves the building.
- Enrichment. A quotation has to appear word for word on the page it is taken from. A chart may only use figures that appear in the source it cites. Whatever fails is dropped, and the reason is kept.
- Fact check. A last pass hunts for claims the article makes and its sources do not.
- A human stop. Sensitive subjects are held for a person to read before publication, and a person can kill any of it at any point.
If that sounds less like a newsroom and more like a factory: quite. It is called Press Factory.
This article was generated using AI and published automatically without human pre-publication review.
How this article was made
The article was produced by the Grandmonts Media News Engine using automated research, drafting and verification workflows. No human editor reviewed the article before publication. Grandmonts Media remains responsible for the published content. Errors can be reported at office@grandmonts.cz.