God I wish they'd back up all of those claims by offering a subscription of Kimi K3 and GLM 5.3, not some outdated GLM 4.7 instance that they then proceed to call a preview model and say that they'll remove it, leaving users only with GPT-OSS 120B which is nigh useless nowadays: https://support.cerebras.net/articles/9996007307-cerebras-co... and https://www.cerebras.ai/pricing
Guess they don't care about regular devs atm and are focused only on hardware sales.
Why would they offer a coding subscription and start competing with some of their biggest customers; when they are capacity-bound and companies like OpenAI will take however many wafers that Cerebras sells to them?
OpenAI's Sol ultrafast (powered by Cerebras) is still in preview, presumably because they're overall capacity bound.
> Why would they offer a coding subscription and start competing with some of their biggest customers; when they are capacity-bound and companies like OpenAI will take however many wafers that Cerebras sells to them?
Because they already have / had an okay coding subscription product for a bit and it gives them visibility and mindshare (in regards to their hardware, even if they don't compete with other providers that much). They could do what Kimi did - make a good subscription with good models, once you get enough customers to get some good PR and such, pause the signups so you don't have to spend more on running the service than you want/can. Do enough of that and people will talk about your offerings organically, make yourselves known to even devs as "That one company with their own hardware and the super fast subscription." experiencing which would do more than any marketing.
Coding subs are good when they promote usage and adoption of your models in enterprises at API rates.
Cerebras is a B2B hardware company. It feels like a distraction: think of the opportunity cost, and resources/headcount not working on other things that would drive more impact.
Should NVIDIA do a coding subscription too? I'm sure they can make money off it, but I think it would be -EV.
In their case not to gain mindshare or money or whatever, they're already a market leader, but to run something that validates the use case of their own hardware (across a bunch of 3rd party models) on a practical level and gain whatever insights or details might be relevant to pass on to other hardware and software teams.
Ultimately the frontier labs are competitors of Nvidia. There is a fixed amount that the market will pay for tokens. If the frontier lab model premium collapses due open weights models, Nvidia can capture a greater share of aggregate spend.
I don’t think they will until they change the architecture.
They don’t have a prefix cache like other providers, or at least don’t have a discount in their billing structure. Each message charges for the whole context window. It’s wildly more expensive for long multi turn scenarios with lots of tool calls (coding). It’s better for short few turn tasks.
Edit: I don’t know if they actually have a proper cache. This could just be a billing artifact.
>Guess they don't care about regular devs atm and are focused only on hardware sales.
Why would they want to target regular devs right now? If they sold to regular devs instead of enterprises, the complaint wouldn't be about model choice, it'd be about how expensive it.
Seriously, even as well paid as many devs are these are not machines that are affordable for personal use. Their market is people slapping down millions on frontier model training.
I still think that was a really great model that got overlooked. It was really great in terms of latency/throughput while still being fairly intelligent.
I was planning on using it for a design tool, but moved over to luna since it's comparable speeds and cost for a lot more intelligence.
> I was planning on using it for a design tool, but moved over to luna since it's comparable speeds and cost for a lot more intelligence.
Everyone should occasionally go back to the old models to see how much worse they were, like even a year ago you could generate results but they were typically full of bugs and you have to fix a non-insignificant amount of it all manually: https://blog.kronis.dev/blog/i-blew-through-24-million-token...
Admittedly that post was before agentic development truly took off and that 3k EUR figure when paying per API tokens would nowadays be closer to like 6k EUR for the volume of work I do, but still.
It's the same how Qwen 2.5 was pretty problematic for anything remotely serious, same with Qwen 3 Coder Next (80B), and at least the most recent versions are getting better but still not quite good enough in real world use cases outside of benchmarks. They've come a long way, regardless!
>Everyone should occasionally go back to the old models to see how much worse they were, like even a year ago you could generate results but they were typically full of bugs
Oh yeah, I'm still amazed how good the current iteration of models are for coding (I have a fear it's too good to be true - so will get taken away..). Exactly a year ago I switched from GPT 5 to Gemini just because the coding with R language was terrible; and even with Python it kept forgetting and mixing basic stuff. Gemini at the time had much longer context window and was miles ahead on R syntax.
Current experience of just leaving a Codex Agent chug until a stable solution is completed is still mind blowing to me.
gpt-oss-120b is absolutely unusable over Cerebras. It fails to call tools half the time and just continues to think about what tool it'll call repeatedly. Like it says it'll call a tool and then it doesn't, and then it says it'll call the tool again and then it doesn't, and it just does that in a loop forever. It's awful. Also forgets to end the thinking block too. Even if the model itself was just-okay for its time, even at 1000t/s+ it's not worth it. And it's EXPENSIVE, like $5 per minute expensive
As MoE with 5B active parameters it's pretty fast. But you still need a lot of vRAM, or have to run small quantitations. Qwen models just gave you more bang for your buck, and the gap became worse with every qwen release
Why they should go with Chinese models if they have a line up of gpt models and a very good partnership with someone who lives in the same jurisdiction and not in the country that convinces their citizen that it’s a good idea to go on war with western world ? Just curious ?
Because they generated some buzz and are near-SOTA and would be a great benchmark for a PoC subscription that doesn't necessarily aim to compete with other vendors at a similar scale (since their main business is the hardware). Mistral is conceptually cool but is lagging behind. I guess Muse Spark and Laguna would also be okay, just not as recognizable. Meanwhile both Kimi K3 and GLM 5.3 are near-SOTA in performance and considerable in size, a great choice for proving the platform!
As for the 2nd part of your question - that wasn't a relevant concern or consideration here, unless the models would be tainted to a degree to prevent them from having a good coding subscription that gets more developer mindshare towards what their chips can achieve and generate some good PR.
So they should go with near sota instead of SoTA due to hype
? In the Current market there are vendors who are ahead and Chinese quickly catch up.
You pick the vendor who is ahead, create an agreement with them to get access ahead of public release , and bake that model into hardware , because that’s how you make money .
You cannot infer this because they only show the tokens per second per user. One way to get a higher number is to have fewer users per chip.
I'm pretty sure Cerebras has a confidentiality agreement with OpenAI, and this press release was carefully constructed to avoid leaking details about the model weights. For example, the graph of tokens per second vs. tokens per second per user doesn't have any numbers that would allow you to translate between the two. (And in any case the relationship depends on the model.)
This was also interesting: "CS-4 delivers more than 1,000 tokens per second on models exceeding 10 trillion parameters." Was it known that there were 10 trillion parameter models in use?
I think the frontier providers keep the size of their models carefully hidden.
You don't really need to train a 10T model to test cerebras against a 10T model. You can feed it an untrained (randomly initialized) model and benchmark it. Result will be gibberish but performance the same.
I'm confused. I thought Mythos 5 and Fable 5 were exactly the same model just with a different security layer in front of it.
Could they mean the Mythos 5 Preview?
> I believe this report has confused Opus (which is known to be around 5T) and Fable.
5T for Opus feels quite high though. DeepSeek V4 Pro is a mere 1.6T and often described as a match with Opus in overall quality. Even the largest open models in common use are around 2.8T.
If this is true, it's even more impressive that some of the open weight models that are <3.5T in size, approx 33% of its size, are within a few points of it in the artificial analysis leaderboard.
Not necessarily, there could be diminishing returns on mere parameters count .
There is nothing to say for example a 1 Quadrillion parameter model will be vastly more intelligent than current SOTA especially since new training data is largely synthetic today
I read it as it is impressive because smaller models 2.5T are squeezing similar returns as 10T models despite being 1/4th size not that there beyond 2T today the number or parameters do not have much meaning
I do not rely on any LLM of any size for general knowledge baked into the weights, they all hallucinate and that is the wrong way to hold them imo
I think there is some merit in that smaller models cannot memorize so much of the training data, i.e. that they are less likely to do copyright infringement, and by analogy not having memorized SDK / API surfaces that have since changed from the training data
> I do not rely on any LLM of any size for general knowledge baked into the weights
You have to rely on it to a certain level for agentic/coding work, presuming that's the general subject we're talking about here... For instance I recently encountered a project where it would have been a lot worse if the LLM didn't already know "what is" xterm.js and a bunch of its associated npm-related/node related software. If it was still smart but had to google and find results for everything it would have been a lot more time consuming and risked sending it down a wrong path.
for sure, there is a minimum size and knowledge base that is required to be useful
at the same time, search may find newer or better alternatives, and you can always specify specific technologies you want to use, I typically do this when starting a new project
But these labs distill off the larger models. Both officially at the labs with the big ones, and unofficially. We need the giant models to get the smaller models.
The fact that Musk claims Opus is 5T to justify why Grok is far behind should be taken with a massive grain of salt given he's a recidivist mythomaniac.
Honestly if Opus is 5T parameters while being matched by the biggest open models that are at least twice smaller, it would mean that the US is already behind China in the AI race, despite a significant edge in compute.
Even if they don't match current-day Opus in everything, they do beat 6 month old Opus, which we have no reason to believe it was smaller than the latest version.
Yes. And Opus goes a very long way compared to Fable, Anthropic isn't doing any favour, it's clearly just 2 models with a very different amount of parameters.
The raw margins on proprietary model inference are rumored to be quite high though (they have to successfully defray the entire investment into model training and datacenter capacity for inference, which is massive enough). The API cost you're paying for the model includes that raw margin.
> The cost to train and infer that would be insane, even by today's standards.
This assumption is likely what has led to the erroneous failure.
Enterprise compute per rack has scaled multiple fold in the last 3-5 years. Alongside the training efficiency gains & datacenter scale increases, even 50T+ is well within reach at the top end.
pretty sure 10 trillion parameters is now the norm among closed ai labs, given that nvidia also references the same 10 trillion number for their nvl72 racks
Memory capacity on the WSE is the same as before, but access to off-wafer memory is much slower, so the sweet spot is a given fixed balance of memory and compute. They have announced a partnership with AMD in which CPU/GPU hardware is used for part of the workload and the WSE-3 machines are used for inference for specialized smaller models, but I'm not really sure of the details on that.
And there is, of course, the educated guesses about what WSE-4 will be, one being adding a LOT of stacked SRAM or DRAM to tip the balance towards memory (which could also be done by having a few different tile designs with various configurations of compute and memory capacity). I am curious about which way they'll go.
AMD along with cerebras may probably compete with NVIDIA monopoly in near future. Also, NVIDIA will have competition form multiple companies. Just my prediction.
Maybe, but GPU is just one aspect of NVIDIA's dominance. If you are buying Vera Rubin GPUs, you're getting an NVL72 rack, which is only one of several racks that you're probably buying. You'll also need your NVIDIA racks with NVIDIA networking & storage gear, too. At the end of the day, they're "vertically integrated" for your accelerated computing data center (e.g. the "AI Factory"). This doesn't even count the software layer, where CUDA + CUDA-X (not to mention the software for all the sysadmin pieces) has a huge first mover advantage over anyone else.
If you are making a decision to spend 50B on hardware, would you use the proven tech stack or rely on engineers taking an unspecified amount of time vibecoding your software stack while the hardware sits idle?
How about in a month or so when you have to run a slightly different workload?
Hyper scalers like Google or Microsoft, which are the big spenders, have all the incentives in the world to get more out of their gargantuan spending.
In fact both of them, actually Amazon too, invest in their own inference hardware and owns the stack.
You can't possibly think that these companies will keep shelling 50-100B per year in hardware alone where 60%+ is margin for Nvidia and not invest there.
if you are developing your own hardware, you provide your own stack to avoid lawsuits with nvidia. i don't think it's a technical problem at all, but a legal one. this is probably why zluda was scrapped by AMD and Intel. Nvidia technically bans the creation of CUDA reimplementations in their TOS if i remember correctly
I don't know then, amd and Intel decided to get rid of Zluda before that lawsuit ruled that apis are fair use. And now, their cloud customers are ok with using their own stack such as rocm, and Intel got rid of their Garuda ai chips. Those things also happened before the lawsuit was settled
It would be a breach of contract not a copyright issue.
It can definitely create a software stack for you if you hold it right, but the software stack supported by a trillion dollar company with decades of expertise, that also uses AI to improve its stack is probably gonna be better.
On the reverse side, there are fundamental limits to the number of ways you can perform certain actions, and agents are both diligent as well as able to swarm. If you have your tests beforehand, there is a chance.
Press doubt. Single GPU? Maybe. MultiGPU behemoths like NVL144 and NVL576? I don't think so.
NVLink is at gen9. they had a lot of teething problems and can codesign the hardware and software.
in the name of openness (AMD's only """weapon"""), the UALink spec is a hodgepodge of corporate opinions with very different implementations (looking at you, Broadcom). at spec version 1 (in hardware).
I wish them good luck as I really like AMD, but they compete no more on this than Lambo vs Bugatti.
Because that would reveal their edge to investors, or the lack thereof.
If Fable turns out to be a 10T or 20T model, there is little to boast vs Kimi at 3T. But the opposite is true: if Fable were to be e.g. a 500B model, that would show how far ahead they are from the open models. This isn't likely to be the case ...
I guess there's also the economic aspect. It would make it much easier for competitors to figure out your costs and margins if they know the model parameter sizes you operate.
Just a reminder for everyone that we are only several years and 3 or 4 iterations into hardware being optimized for LLMs. We should all expect orders of magnitude improvement in speed and/or cost over the next 5 years. Then we can have fun conversations about "unlimited" "intelligence" and about what the price wars and profit margins of consumer AI products are when your average ChatGPT user costs the company $0.10 per month.
> CS-4 delivers more than 1,000 tokens per second on models exceeding 10 trillion parameters
And, the software side isn't finished being optimized, either. We've seen with Qwen 3.8 27B and DeepSeek V4 Flash 0731 and GLM 5.3 that quite small models can pack a punch. Intelligence density will improve, efficiency of kernels will improve, efficiency of KV caching and MTP will improve, algorithms for splitting workloads across compute units will improve.
It'll all be as cheap as DeepSeek was before the price hike. And, it'll become more and more realistic to run near-frontier intelligence on personal devices.
Taalas will be one of the great disaster investments of the early AI era. It'll be a near total write-down.
The absolute worst market time to etch a model to a chip is right now (very rapid iteration). There is no scenario where they can keep up. The Taalas approach will be viewed as comically foolish within just a few years.
Cerebras will win in terms of approach.
It's 1998: hey, I can drastically speed up your web service, let's etch it right to silicon.
1) waiting for another 3-5 generations of transistor improvements before it can fit into a single conventional chip, or
2) another generation before getting a monster of a chip (1000+ mm^2), and prices for flawless etching scale quadraticly (likely $1000+ for manufacturing costs alone).
Could happen, but it's a long shot for a market that could be satiated by specialized accelerators.
In Taalas HC2 a chip embeds 20b parameters, and the declared idea is linking the chips. A card with two of them chips and you can already have a dense Qwen at staggering speeds.
That is thinking about an LLM (logical) producer and server. For mass production, the costs go down. And in the case of a ~100b model as the poster mentioned, they would be just single PCI cards with 5 or 6 HC2 chips: doable and practical.
I see more potential problems in the positioning of the SRAM - but not a real problem given that excellent team.
To get a proper idea of the costs the architecture of the HC2 will have to be clearer.
> It's 1998: hey, I can drastically speed up your web service, let's etch it right to silicon.
I distinctly remember 32-bit/33 MHz PCI accelerator cards for SSL being a real thing (for use on OpenBSD or FreeBSD), in an era when something like a single core 700 MHz Pentium 3 1U system was a relatively powerful individual bare metal httpd box.
The CPU load of doing a lot of SSL purely in software was a problem in terms of scaling things up, so this was one attempt at a (very short lived) solution. Note that this predated TLS1.0.
> The absolute worst market time to etch a model to a chip is right now
Slightly disagree. It really depends on the price-point at which they can do that etching. ~1k usd / ~30B model in a hdd-sized case that fits on your desk? I'd buy one right now, even knowing that I'm "stuck" with whatever model of the day is.
Time to market also matters a ton. If they can start shipping chips <1 month after the weights drop that's much more compelling than if it's a 6+ month development pipeline.
In the case of needs to process natural language, instead, massive efficiency (esp. time) can be a game changer. It's like "you have two years to complete the project" vs "you have two hours to complete the project": if you can squeeze that "two years worth" into a negligible delay, it's a game changer.
Yes, power efficiency may be the largest benefit of these chips actually - especially if the projections are true that the US and other countries simply aren't able to ramp up power generation to meet forecasted datacenter demand.
There's a bit of a ticking time bomb there, something that a Taalas-like architecture can clearly resolve.
maybe AMD wants the IP to deploy it once ai model development slows down in a few years. Or, their large cloud customers do want to burn through silicon, basically paying rent to AMD for models etched on silicon.
I just want to but hardware so I can run a model at home that is fast. I don't see myself installing a server that burns almost two hundred kilowatts but maybe a card which runs a 27B Qwen...
> Then we can have fun conversations about "unlimited" "intelligence" and about what the price wars and profit margins of consumer AI products are when your average ChatGPT user costs the company $0.10 per month.
We can have that discussion now: sounds like that would kill OpenAI and Anthropic
A design that bakes the architecture into silicon would be 10x faster, and imagine a version that does all the multiplication ops using single log-amp addition versus dozens of transistors to cut down the amount of silicon used by 50x. The ceiling for AI optimized hardware is extremely high.
Stack on top of that the fact that diffusion based models like the ones made by Inception Labs are far faster and more efficient than autoregressive LLMs and have an even higher ceiling of optimization (single step path prediction via model distillation versus 50 step denoise is currently an active area for image diffusion)
The human brain is soon neither going to be more powerful nor energy efficient than the stuff we use to run AI.
This is part of why I think the data center build-out is a bubble. We've barely scratched the surface when it comes to hardware optimization. We'll see exponential improvements in energy efficiency and speed over the next decade. Exponential, not linear.
GPUs really aren't that great for AI. They just happen to be the best chips we have in mass production right now for this work load, and it takes time to field new designs. Basically every chip engineer on the planet is working on this right now.
Whether it's a bubble or not depends on how much the demand for compute and the type of workload keeps growing, though.
If AI tends to be something used mainly in ideation and development, which is how a lot of people use it today, then once consumer hardware gets good enough you could see a bunch of the current data centre workloads move onto consumer devices.
But if AI starts being used more in repeatable, operational workloads I think it makes sense to have significant cloud infrastructure for it. TBH I haven't seen much of this, and I've been skeptical about people using agents for much of anything when it can be done with just software. But we are starting to see more of this kind of workload, like the taggable Claude in your slack etc that people seem to really love.
By the way, this is the same argument that Michael Burry used to short Nvidia.
He claims that GPU depreciation/obsoletion is much faster than hyperscalers are assuming because new chips will be much better. He's being proved wrong right now because H200 rental prices have been claiming for the last 8 month despite B200 having 10-20x better inference efficiency.[0]
The logic is fundamentally flawed in my opinion. Let's use future Nvidia chips being much better optimized for LLMs for example.
New Nvidia chips 10x better than H200 --> data centers buy a lot --> Nvidia profits a lot.
New Nvidia chips 10x better than H200 --> data centers don't buy --> no faster than expected obsoletion.
In other words, the very act of buying many new Nvidia GPUs would be the event that causes faster than expected obsoletion. Yet, if you don't buy those new Nvidia GPUs, then there is no faster than expected obsoletion.
We also live in a world where there is competition. If Amazon doesn't buy but Microsoft does, suddenly Microsoft can offer better $/token prices.
1. The same isn’t necessarily true of the rest of the hardware stack which may be reused between accelerator generations.
2. You’re missing the “New Nvidia chips 10x B200, compute requirement grows less than 10*software improvements YoY -> buy less Nvidia.” Valuations are based on forward projections (>1T annual for NVDA) which can be revised down leading to a drop in valuation.
> If Amazon doesn't buy but Microsoft does
The big 3 all have their own proprietary accelerators. Meta is buying TPUs as well for now.
I would bet Nvidia’s major customers in 2 years are neoclouds and it seems that Jensen is making the same bet.
1. So this makes Burry’s argument even less convincing since those auxiliary hardware can last longer.
2. Jevons Paradox. More efficiency should lead to bigger models, faster inference, and more total tokens.
3. By all accounts, Trainium and Maia and Meta’s internal chip are struggling to keep up with Nvidia. That’s why they order as many Nvidia chips as possible. They’re not giving up but it isn’t as easy as buying stock Arm cores and taking them to TSMC.
Neoclouds may very well be Nvidia’s biggest customers and this probably what Nvidia wants.
1. Not really, current valuations are priced for persistent 80%+ margins based on spot. If auxiliary hardware lasts longer (I.e. next gen GPU reusing the same shell) then that reduces supply pressure and spot prices.
2. Jevon’s paradox is about total consumption, not margins. Valuations are about margins (and their projections). Many coal mine owners went bust despite increased total coal consumption.
3. Source? Gemini for example is 70% on TPU. I have yet to see data on Maia-300 beyond Microsoft PR. Remember it doesn’t have to be better it has to be more cost efficient. The overwhelming majority of inference spend does not care if token output is 20% slower if it is 50% cheaper.
> Neoclouds may very well be Nvidia’s biggest customers and this probably what Nvidia wants.
What Nvidia needs. Whether neoclouds can stay competitive vs hyperscalers paying Nvidia tax is far from clear, particularly when inference margins compress.
1. The whole Burry argument is that AI hardware becomes obsolete faster. If aux hardware can be reused, that works against the argument.
2. Total consumption drives more demand for the already supply constrained hardware. Can AI hardware market go bust? Sure it can. But being early is the same as being wrong in the investment market. When do you predict the bust to be?
3. Google, Amazon, Microsoft, Meta are all buying as many Nvidia GPUs as they possibly can. The biggest tell on how Nvidia is doing is that their share in inference has increased despite the increase in competition: https://archive.md/CKP0N. So while competition is getting bigger and bigger because the overall pie is getting exponentially bigger, Nvidia's growth is still higher than average.
> 1. The whole Burry argument is that AI hardware becomes obsolete faster. If aux hardware can be reused, that works against the argument.
Burry’s main argument is depreciation is being understated and the capex vintages will not be paid off before they are essentially useless. This can happen whether or not aux is reused.
> Total consumption drives more demand for the already supply constrained hardware.
Demand is the wrong metric.
Only number that matters is whether AI attributable revenue will be sufficient to pay back enough of each successive capex vintage (e.g. 750B this year, 1T next year, 1.2T in 2028) so that hyperscalers and neoclouds can either self-fund or continue to issue debt as bond markets are already straining and tax-payer backed sovereign debt is providing a high baseline. Otherwise they downgrade capex projections and the bubble pops.
Expensive compute needs expensive inference to justify 30-40B/year/GW of compute. There are many reasons why frontier API pricing which is what the industry is based on may not persist. It is also almost certainly the case that 2026 is the worst year of supply and demand mismatch to allow for 80%+ margins. HBF next year has the potential to single handedly pop the DRAM spot bubble.
> Can AI hardware market go bust? Sure it can.
This is the bear thesis. It is not that AI will crash or be useless.
> But being early is the same as being wrong in the investment market. When do you predict the bust to be?
Q4 27-Q2 28 is when the bill becomes due at the latest. There are sufficient financial levers left to buy time without returns until then.
> Google, Amazon, Microsoft, Meta are all buying as many Nvidia GPUs as they possibly can.
All of these companies have rock solid revenue streams and can easily swallow 500B of capex devaluation over time. Their buying of Nvidia today is not necessarily the indicator you are implying as there are strong competitive reasons to make the game more expensive for everyone else.
Burry’s main argument is depreciation is being understated and the capex vintages will not be paid off before they are essentially useless. This can happen whether or not aux is reused.
And why does he think depreciation is understated? It is because he thinks newer Nvidia GPUs will make older ones obsolete faster. Hence, my entire post.
The rest of your argument centers around whether AI growth will meet the cap ex expenses. I don't see anything new in it.
HBF next year has the potential to single handedly pop the DRAM spot bubble.
I'll believe it when I see it. Jevons paradox will apply here again in my opinion. HBF does not replace HBM.
A cursory estimate courtesy of ChatGPT suggests that there is a grand total of one order of magnitude or less of power efficiency improvement available compared to current Blackwell if the entire system’s power consumption outside the ALUs went all the way to zero.
If you want three orders of magnitude improvement, you probably need to find two of those orders of magnitude somewhere else: process improvements, different ALU design, model architecture changes, etc.
True, I have to agree with you. The AI giants might be investing a huge amount of money in generation 1 technology. There might be a much better way to do it just around the corner. They might know this and thus the hurry to IPO.
A rough analogy would be if the first generation of ISP's spent billions on dial-up exchanges, when fibre could be invented next year.
What LLM-specific hardware improvements should one expect? Seems to me that LLM inference is simple architecturally (matmul et al) so most scaling in hardware should come from general improvements (memory BW, packaging, interconnect, power).
What you describe is basically Cerebras case, at the bottom it's just a really big die (about x28 an NVIDIA GB200) with a lot of work to reduce memory latency and improve throughput.
What it's actually amazing is how can they make a chip so big and still have a decent yield to be commercially viable.
Congratulations! You have just realized that the AI data center build out is a total scam, built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling.
There exist other AI accelerators (TPUs, ASICs) that perfectly exceed the throughput that LLMs need to scale as well. But the true solution is more software optimizations. There's a tiny handful of them but more needs to be discovered so that we can reduce building hundreds of more data centers as the alternatives mature.
As better software becomes more useful for the alternative AI hardware for developers with LLMs running efficiently you then would have more choices of hardware to run your LLMs on rather than just only GPUs.
TPUs and ASICs run in data centers too. Your argument only holds true if there's some satisfied limit to demand for inference. If not, data centers will continue to spring up to host more and more agents. Even if agents were running on hardware and software as efficient as the human brain, its conceivable we want trillions of them running at any given time which would require data center scale.
Everything has some satisfied limit to demand, often depending on the price. If you assume there will never be any satisfied limit to demand for inference at any price you can justify any investment.
Yeah, but there's certainly a part of the curve where price drops by X OOMs and demand increases by much more than X OOMs. (Presumably some of that is substitution and some of that is new use cases.)
At 75.3 trillion tokens for the week ending 10 Aug 2026, that means that up to 450 trillion tokens were plausibly demanded by the whole market for that week.
My take: At max saturation, each person on earth could have their demands satiated by an average of 16 agents running concurrently. Sometimes more, often times less, but the average would likely be at 16.
At 200 tokens/second for each agent, that would mean 15.48288 quintillion tokens per week.
We're currently at about 0.00290643601% of the calculated demand ceiling.
Even if the demand limit per person is just 1 agent at 50 tokens/second, the current demand's still 0.186011905% of the theoretical ceiling.
I wonder what this looks like in 5 years... Will there be a massive push to repurpose these giant boxes into housing? Will they get turned back into the farm land from where they came? When a data center goes bust, what happens to the parts left behind?
I'd think the infrastructure would tend towards factories, smelters, and so on. Industrial things that have reasonably high power demands, can use the building, and don't care about the lack of windows.
They're typically not built where you want housing, and the buildings are distinctly the wrong shape.
If you can't use the power infrastructure profitably my next thought would be warehousing.
But also... we've seen a pretty continually increasing demand for compute. Even if AI busts a bit (or becomes a bit more efficient) I bet most data centres stay data centres, just less profitable ones.
Huh, why I'm not surprised that HN is full of opinions confidently stated without any numbers or resources to back up?
> built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling.
Insurmountable according to whom? And who assume that only GPUs are all we need to continue scaling? Google, Amazon, Microsoft, Meta and OpenAI, all have or plan custom non-GPU AI chips. Do they plan to use them not for scaling?
Information about RAM type/size and connection topology of the RAM to be used for context cache seems to be conspicuously absent from the slick looking marketing materials.
44GB on-chip-sram * 3 chips. Per chip: 43.2 PB/s memory access + 53.5 PB/s on-chip fabric bandwidth + 2.4 Tbits/s "IO" bandwidth (I think that means their RoCE v2 RDMA over Ethernet interface).
I suspect there might be a certain amount of customization for how much RAM they attach when you order it.
They have managed to make the external link 300GBps/2us. Cs3 was 150/5.
This is 1/3rd blackwells nvlink c2c bandwidth already. Not too bad. We can make KV cache offload work with that I suppose.
If magically KV cache was not an issue, pipeline parallelism on cerebras can be quite pleasant. As for the KV cache offload, I have hopes their CPO solution they're trying with that canadian company ends up bearing fruit.
If cerebars is performing well, why didn't its predecessor, server S-3, become the largest API token provider on OpenRouter, surpassing the official model releases?
Without having any inside information, one possible theory:
All or a vast majority of of the cerebras manufacturing capacity was going to a few companies that aren't publicly available inference providers on openrouter, for their own internal use.
or
The asking price of the S-3, no matter how speedy it might be, for small/medium size customers made it economically prohibitive to purchase and use to sell public inference vs. buying more common nvidia b200 or whatever.
But that's supply and demand, not technology. Right now a lot more people want their inference than they can supply. as supply catches up in the next 5-10 years, the underlying tech at scale is probably cheaper than GPUs per token produced.
it only takes ~445 GB300 NVL72 (about $22b) to run ALL of openrouter demand for a year. Microsoft rolled out $32b of DC 2026Q1.
imo the issue is that most openrouter demand is inauthentic activity (things that anthropic and openai models will refuse to do like pretend to not be bots when interacting with humans)
So I plugged 288 trillion tokens/month (OpenRouter's current rate), 500 billion MoE model average, and the math comes out to be around 620 B200 GPUs minimum.
So basically, OpenRouter's volume must be absolutely tiny compared to the volume hyperscalers are getting.
I love how instead of comparing a Ferrari (fast and expensive) to some average car (not fast, not expensive) to make your point..you went for public transport where your comparison cracks from multiple angles.
Because comparing it to an average car is wrong. It isn't about fast/expensive. It is about fast/capacity.
A Ferrari can seat up to 4 people[0], which is about the same as the average car. Capacity doesn't change much.
Meanwhile, a bus/subway system is meant to support millions of people. Tokyo's metro has to support up to 37 million people. You can't do that with Ferraris.
Because they aren't selling inference, they're selling hardware. The only reason they sell any tokens on OpenRouter is so they get on the benchmark that shows them as the fastest provider. It's free advertising.
I use Cerebras via OpenRouter. It’s every bit as fast and reliable for my needs as claimed. I suspect the reason is that they can either be making peanuts selling inference to plebs like me via OpenRouter, or making bank selling the more expensive models to businesses directly. In short: I would be very surprised if they have die capacity, and are at this point maximising revenue per chip.
And advanced geothermal. Fervo Energy let's us get energy that's not based on burning fossil fuels but is, instead, able to produce energy from the ground.
The core is obviously not in question here, but at scale and long term it could have an impact on the water bed, vegetation, underground ecosystems, soil stability, etc.
Actually that's mostly just the military funding those, with a few of them having data center partnerships so they can shield themselves from the criticism of what they really are: military contractors.
I heard a great deal of noise from early 2002 to the present date that the US military had a high interest in small portable nuclear reactors for large bases in Iraq and Afghanistan. And particularly around the peak period of troops on the ground in AF and IQ. And indeed a place like Bagram or Kandahar used a shitton of diesel to run generators. But nothing ever came to fruition to actually implement it, has something changed now that they actually consider it worth doing?
Distribution's always the problem - how much power actually gets delivered to each rack. 162 is ~an order of magnitude higher than normal, which means nothing in a standard data center is going to be built to deliver that kind of power to one rack.
Indeed. You need 45 to 60 liters per second of cooling water flowing over a Cerebras wafer every minute to keep it under 90C. And that’s assuming the water leaves at 90C…
More realistically, you need much more cooling water.
Impressive that this is an "interim" product, the start of a new line that ought to be continued with the WSE-4 family, where they are supposed to use a 3nm process and, maybe, 3D stacked SRAM. The modular architecture also points towards field upgrades that are badly needed for AI datacenter builders.
Interestingly they’re still on the WSE-3 (5nm TSMC) wafer chip and slightly bumped up the specs there (overlocking mostly it seems), for why it’s called WSE-3 Turbo now. I think people were also expecting WSE-4, as it’s been 2 years now since WSE-3 was launched.
They do offer API services to individual users... though with a set of models that makes it unlikely that you want to use it. They are promising Qwen 3.8 27B any day now though*.
if you have the money as an "individual user" to purchase one of their racks... save your money and retire.
* Actually they sent out an email claiming they already have it, but I don't seem to have access, they're promising to release it to the "shared tier" any day now.
Now that this hypothetical person has retired, what are they gonna do all day? Just sit on the beach and drink Mai Tais? If that's what they wanna do, sure, but nerds gonna nerd, and if I had that kind of money to retire on, I'd totally buy some ridiculously expensive AI box for fun.
Ah but if you have the kind of money where this is a reasonable retirement hobby purchase, you aren't bothered by representing yourself as a "enterprise" :P
The Cerebras hardware is not locked to specific models / model families. Taalas is the company that's etching models into their silicon, locking it to that model forever.
there is an extensive and complicated cooling system that permeates the wafer. some of the cores are completely turned off because they fail qc (see the tsmc logo) if all of their neighbors have been going at full bore the thermal differential can cause stress fractures if the cooling system suddenly fails.
Five years from now, I don't know why anyone will still be using Nvidia for inference. Note that Cerebras is for inference only, not for training.
I understand that Cerebras has competition, but this bodes even more poorly for Nvidia for inference. Nvidia may still have a role to play for training, however.
NVIDIA has the best supply chain in the entire game. They are the only ones who can produce at their scale. You really shouldn’t underestimate their position
Nvidia is at this time a pretty well run company tech wise. They are going to keep iterating on the inferencing hardware stack over the next five years too.
OpenAI needs to immediately move to acquire Cerebras.
Nvidia's extreme margin is the opportunity for OpenAI's cost reduction. Buying Cerebras would pay for itself and they should take all of its future production (after filling required contracts).
Right now China's models have no silicon moat. Cerebras as a drastic speed-up / cost-reduction potential, can assist in building a competitive moat. And every time a Cerebras pops up, OpenAI or Anthropic should eat them if at all possible.
There's no stand-alone frontier AI company of great scale in the near future that doesn't have a large silicon advantage in-house. Apple knew it in smartphones, Google figured it out a long time ago as well.
OpenAI is partnering with Cerebras while simultaneously investing in their own silicon play. Hedged bets.
After sitting thru their keynote today, it makes sense. The main throughput speedups they tout are an obvious evolution of the GPU that all companies will be building in the next year. Wafer-scale interconnected memory and compute is just going to beat out mountains of network cabling any day on both cost and performance metrics.
> Introducing the all new Cerebras CS-4, a revolutionary rack-scale solution that delivers upto 30x faster inference compared to GPUs, enhanced economics, and a simple path todeploy [sic] hyperscale capacity.
If they had ask Claude it would probably look like this: Introducing the all new Cerebras CS-4, a revolutionary rack-scale solution that delivers up to 30x faster inference compared to GPUs, enhanced economics, and a simple path to load-bearing hyper scale capacity.
The comparison seems incomplete. CS‑4 is a full rack-scale system with three wafer-scale processors, but the exact GPU models, GPU count, power consumption, price information are not disclosed.
We still don't know if buying a multi-GPU rack (or racks) is cheaper and/or more efficient in power.
The fact that they didn't disclose these numbers makes me believe that the numbers are not in their favor. And personally, makes me see them as disingenuous.
Cerebras should slowly also move to dgx/ryzen market for a desktop version for masses at affordable price yet providing substantial tokens/second on desktop
Is it just me or is it bizarre that they're advertising old open-weight models.
GLM 4.7 (December 2025) not 5 (Feb) 5.1 (April) or 5.2 (June). 5.3 (4 days ago) is, to be fair, not open weights yet... but there's a lot since 4.7.
Kimi K2.7 (April) not K2.7-code (June) or K3 (July).
Gemma 4 (April), Llama (April), and gpt-oss (August 2025) are up to date, but old (for models).
Meanwhile the closed source GPT 5.6 sol is up to date (June)...
Should potential purchasers take away from this that they're not going to be able to run recent models unless they front the cost of developing software or something?
I mean the product is a server rack and while there's no advertised price I would assume it's six figures. So yes, an enterprise product.
But even an enterprise is going to care about the difference between "we can run the model we want with support from the manufacturer" and "we have to purchase the product, and then spend another 6 figure sum having developers port a recent model to the product to use it".
God I wish they'd back up all of those claims by offering a subscription of Kimi K3 and GLM 5.3, not some outdated GLM 4.7 instance that they then proceed to call a preview model and say that they'll remove it, leaving users only with GPT-OSS 120B which is nigh useless nowadays: https://support.cerebras.net/articles/9996007307-cerebras-co... and https://www.cerebras.ai/pricing
Guess they don't care about regular devs atm and are focused only on hardware sales.
Why would they offer a coding subscription and start competing with some of their biggest customers; when they are capacity-bound and companies like OpenAI will take however many wafers that Cerebras sells to them?
OpenAI's Sol ultrafast (powered by Cerebras) is still in preview, presumably because they're overall capacity bound.
> Why would they offer a coding subscription and start competing with some of their biggest customers; when they are capacity-bound and companies like OpenAI will take however many wafers that Cerebras sells to them?
Because they already have / had an okay coding subscription product for a bit and it gives them visibility and mindshare (in regards to their hardware, even if they don't compete with other providers that much). They could do what Kimi did - make a good subscription with good models, once you get enough customers to get some good PR and such, pause the signups so you don't have to spend more on running the service than you want/can. Do enough of that and people will talk about your offerings organically, make yourselves known to even devs as "That one company with their own hardware and the super fast subscription." experiencing which would do more than any marketing.
Coding subs are good when they promote usage and adoption of your models in enterprises at API rates.
Cerebras is a B2B hardware company. It feels like a distraction: think of the opportunity cost, and resources/headcount not working on other things that would drive more impact.
Should NVIDIA do a coding subscription too? I'm sure they can make money off it, but I think it would be -EV.
> Should NVIDIA do a coding subscription too?
Yes, obviously! Well maybe not a subscription but definitely an inference service.
https://build.nvidia.com/
https://resources.nvidia.com/en-us-inference-infrastructure/...
https://www.nvidia.com/en-us/data-center/dgx-cloud-lepton/
In their case not to gain mindshare or money or whatever, they're already a market leader, but to run something that validates the use case of their own hardware (across a bunch of 3rd party models) on a practical level and gain whatever insights or details might be relevant to pass on to other hardware and software teams.
Ultimately the frontier labs are competitors of Nvidia. There is a fixed amount that the market will pay for tokens. If the frontier lab model premium collapses due open weights models, Nvidia can capture a greater share of aggregate spend.
> Should NVIDIA do a coding subscription too?
They sort of do? They offer free access to various versions of nemotron via multiple routing services.
They do have Cerebras Code
https://www.cerebras.ai/code
But it's not fully open to just anyone, I wasted time signing up to find out that I couldn't even sign up for it to test it out.
I don’t think they will until they change the architecture.
They don’t have a prefix cache like other providers, or at least don’t have a discount in their billing structure. Each message charges for the whole context window. It’s wildly more expensive for long multi turn scenarios with lots of tool calls (coding). It’s better for short few turn tasks.
Edit: I don’t know if they actually have a proper cache. This could just be a billing artifact.
Cerebras supports prompt caching and has a doc about it. A fairly standard automatic prefix-based implementation with 5min expiry.
They do not seem to discount cached input for the self-serve Developer tier. Maybe they do for enterprise rate cards?
https://inference-docs.cerebras.ai/capabilities/prompt-cachi...
ah, that makes it feasible! Okay, glad it's not technical limit. They should fix the pricing...
>Guess they don't care about regular devs atm and are focused only on hardware sales.
Why would they want to target regular devs right now? If they sold to regular devs instead of enterprises, the complaint wouldn't be about model choice, it'd be about how expensive it.
Seriously, even as well paid as many devs are these are not machines that are affordable for personal use. Their market is people slapping down millions on frontier model training.
> GPT-OSS 120B which is nigh useless nowadays:
I still think that was a really great model that got overlooked. It was really great in terms of latency/throughput while still being fairly intelligent.
I was planning on using it for a design tool, but moved over to luna since it's comparable speeds and cost for a lot more intelligence.
> I was planning on using it for a design tool, but moved over to luna since it's comparable speeds and cost for a lot more intelligence.
Everyone should occasionally go back to the old models to see how much worse they were, like even a year ago you could generate results but they were typically full of bugs and you have to fix a non-insignificant amount of it all manually: https://blog.kronis.dev/blog/i-blew-through-24-million-token...
Admittedly that post was before agentic development truly took off and that 3k EUR figure when paying per API tokens would nowadays be closer to like 6k EUR for the volume of work I do, but still.
It's the same how Qwen 2.5 was pretty problematic for anything remotely serious, same with Qwen 3 Coder Next (80B), and at least the most recent versions are getting better but still not quite good enough in real world use cases outside of benchmarks. They've come a long way, regardless!
>Everyone should occasionally go back to the old models to see how much worse they were, like even a year ago you could generate results but they were typically full of bugs
Oh yeah, I'm still amazed how good the current iteration of models are for coding (I have a fear it's too good to be true - so will get taken away..). Exactly a year ago I switched from GPT 5 to Gemini just because the coding with R language was terrible; and even with Python it kept forgetting and mixing basic stuff. Gemini at the time had much longer context window and was miles ahead on R syntax.
Current experience of just leaving a Codex Agent chug until a stable solution is completed is still mind blowing to me.
gpt-oss-120b is absolutely unusable over Cerebras. It fails to call tools half the time and just continues to think about what tool it'll call repeatedly. Like it says it'll call a tool and then it doesn't, and then it says it'll call the tool again and then it doesn't, and it just does that in a loop forever. It's awful. Also forgets to end the thinking block too. Even if the model itself was just-okay for its time, even at 1000t/s+ it's not worth it. And it's EXPENSIVE, like $5 per minute expensive
As MoE with 5B active parameters it's pretty fast. But you still need a lot of vRAM, or have to run small quantitations. Qwen models just gave you more bang for your buck, and the gap became worse with every qwen release
GLM 4.7 is gone (at least for us), with no suitable replacement from Cerebras. I think all they care about now is hardware and OpenAI hosting.
Exactly, they need to push to regular consumers as well as businesses. This will create more pressure for adoption
Cerebras is the fastest provider by far on OpenRouter, and gpt-oss-120b is still very useful. They have backed up their claims very well.
> Guess they don't care about regular devs atm and are focused only on hardware sales
They aren't trying to make a few bucks off tokenmaxxers. They're trying to be the underpinning of compute for all AI. They're going to beat Nvidia.
Why they should go with Chinese models if they have a line up of gpt models and a very good partnership with someone who lives in the same jurisdiction and not in the country that convinces their citizen that it’s a good idea to go on war with western world ? Just curious ?
> Why they should go with Chinese models
Because they generated some buzz and are near-SOTA and would be a great benchmark for a PoC subscription that doesn't necessarily aim to compete with other vendors at a similar scale (since their main business is the hardware). Mistral is conceptually cool but is lagging behind. I guess Muse Spark and Laguna would also be okay, just not as recognizable. Meanwhile both Kimi K3 and GLM 5.3 are near-SOTA in performance and considerable in size, a great choice for proving the platform!
As for the 2nd part of your question - that wasn't a relevant concern or consideration here, unless the models would be tainted to a degree to prevent them from having a good coding subscription that gets more developer mindshare towards what their chips can achieve and generate some good PR.
So they should go with near sota instead of SoTA due to hype ? In the Current market there are vendors who are ahead and Chinese quickly catch up.
You pick the vendor who is ahead, create an agreement with them to get access ahead of public release , and bake that model into hardware , because that’s how you make money .
Wouldn't you want to keep this for internal development? Keep the single thread gains for yourself.
Didn't OpenAI bought it?
I think the fun takeaway from this is that GPT 5.4 is probably 45B active parameters and GPT 5.6 Sol is closer to 50B.
You cannot infer this because they only show the tokens per second per user. One way to get a higher number is to have fewer users per chip.
I'm pretty sure Cerebras has a confidentiality agreement with OpenAI, and this press release was carefully constructed to avoid leaking details about the model weights. For example, the graph of tokens per second vs. tokens per second per user doesn't have any numbers that would allow you to translate between the two. (And in any case the relationship depends on the model.)
They show that CS-4 can't really do batching (or rather it can't properly benefit from it), total throughput barely changes (25%?): https://cdn.sanity.io/images/e4qjo92p/production/6a132331880...
Which I think makes it feasible to approximate activation from CS-4 tokens per second per user.
(Where did you see that?)
This was also interesting: "CS-4 delivers more than 1,000 tokens per second on models exceeding 10 trillion parameters." Was it known that there were 10 trillion parameter models in use?
I think the frontier providers keep the size of their models carefully hidden.
You don't really need to train a 10T model to test cerebras against a 10T model. You can feed it an untrained (randomly initialized) model and benchmark it. Result will be gibberish but performance the same.
Mythos/Fable are around 10T:
> According to FT, industry estimates say Anthropic's most advanced Mythos 5 has about 8 trillion parameters and Fable 5 about 5 trillion
https://www.reuters.com/technology/bytedance-targets-mega-ai...
I believe this report has confused Opus (which is known to be around 5T) and Fable.
Other reports say 10T. See for example https://eu.36kr.com/en/p/3760679047267075?ref=explainx where Musk talks about the models being trained on Colossus2
I'm confused. I thought Mythos 5 and Fable 5 were exactly the same model just with a different security layer in front of it. Could they mean the Mythos 5 Preview?
Yes. One reason why I think that report has confused Fable and Opus.
> I believe this report has confused Opus (which is known to be around 5T) and Fable.
5T for Opus feels quite high though. DeepSeek V4 Pro is a mere 1.6T and often described as a match with Opus in overall quality. Even the largest open models in common use are around 2.8T.
> and often described as a match with Opus in overall quality
It's not. Idk about who has more T's but, unfortunately, DS4 pro is not a match to Opus, at least not Opus 4.8.
It's not an Opus match.
The difference is very visible in long tail applications. Exactly where you'd expect parameter count to matter.
It's rumored fable is around that 10T number
If this is true, it's even more impressive that some of the open weight models that are <3.5T in size, approx 33% of its size, are within a few points of it in the artificial analysis leaderboard.
Not necessarily, there could be diminishing returns on mere parameters count .
There is nothing to say for example a 1 Quadrillion parameter model will be vastly more intelligent than current SOTA especially since new training data is largely synthetic today
That's precisely what he is saying, there is diminishing returns (or optimization left on the table).
I read it as it is impressive because smaller models 2.5T are squeezing similar returns as 10T models despite being 1/4th size not that there beyond 2T today the number or parameters do not have much meaning
or the latest qwen3.8 27B doing so well at ~1/100 the size of K3
What about general knowledge you can get out of it before hallucinations start?
I do not rely on any LLM of any size for general knowledge baked into the weights, they all hallucinate and that is the wrong way to hold them imo
I think there is some merit in that smaller models cannot memorize so much of the training data, i.e. that they are less likely to do copyright infringement, and by analogy not having memorized SDK / API surfaces that have since changed from the training data
> I do not rely on any LLM of any size for general knowledge baked into the weights
You have to rely on it to a certain level for agentic/coding work, presuming that's the general subject we're talking about here... For instance I recently encountered a project where it would have been a lot worse if the LLM didn't already know "what is" xterm.js and a bunch of its associated npm-related/node related software. If it was still smart but had to google and find results for everything it would have been a lot more time consuming and risked sending it down a wrong path.
for sure, there is a minimum size and knowledge base that is required to be useful
at the same time, search may find newer or better alternatives, and you can always specify specific technologies you want to use, I typically do this when starting a new project
Storing general knowledge in VRAM has always been a dumb idea in the first place.
It did OK on schlongbench v1.0 (test of a specific niche word that doesn't make it into smaller LLMs) but it sure does love to count words
https://pastes.io/r8F1AY8h
Qwen 3.8 27B beats Opus, Fable and GPT 5.6 by a comfortable margin on the AA-Omniscience Hallucination Rate benchmark.
GLM 5.3 is "only" 753B parameters. Much much smaller.
You want to take a look at the "Scaling Laws" paper, so you can extrapolate from these numbers.
This paper, as well as the Chinchilla one, aged like milk though.
And GLM is only 0.7T!
But these labs distill off the larger models. Both officially at the labs with the big ones, and unofficially. We need the giant models to get the smaller models.
Fable is most definitely nowhere near 10T.
The cost to train and infer that would be insane, even by today's standards.
Fable is strongly believed to be around 10T. The most conservative estimate I've seen is 8T.
Eg: https://www.reuters.com/technology/bytedance-targets-mega-ai...
That reports Mythos as 8T and Fable as 5T, but I think they mean Opus as 5T, which is widely known, eg: https://eu.36kr.com/en/p/3760679047267075?ref=explainx
Both Grok and Bytedance are training 10T models.
The fact that Musk claims Opus is 5T to justify why Grok is far behind should be taken with a massive grain of salt given he's a recidivist mythomaniac.
Honestly if Opus is 5T parameters while being matched by the biggest open models that are at least twice smaller, it would mean that the US is already behind China in the AI race, despite a significant edge in compute.
The open models don't really match Opus.
For example I regularly do Fable+Opus agentic coding runs over 24 hours without intervention.
I think I've had GLM do a run that was a few hours. That's the closest I've had an open model come on that kind of work.
Even if they don't match current-day Opus in everything, they do beat 6 month old Opus, which we have no reason to believe it was smaller than the latest version.
Yes. And Opus goes a very long way compared to Fable, Anthropic isn't doing any favour, it's clearly just 2 models with a very different amount of parameters.
If Fable is seriously around 10T and Kimi K3 sidles up to it at 2.4T
That would be extremely surprising and a massive blunder by Anthropic in model design architecture ... which I highly doubt to be the case.
In real world comparisons K3 is somewhere between Sonnet and Opus. Fable is just a completely different (higher) level.
The long tail of tasks and queries is where you see the difference.
Wasn't Opus ~1.5T and Fable is about twice that?
Kimi K3 is a 2.8T model that's available at about 1/4-1/3 the cost of Fable from multiple providers on openrouter. The math doesn't seem wildly off.
The raw margins on proprietary model inference are rumored to be quite high though (they have to successfully defray the entire investment into model training and datacenter capacity for inference, which is massive enough). The API cost you're paying for the model includes that raw margin.
The Chinese have similarly high profit margins on Inference via their first party api
> The cost to train and infer that would be insane, even by today's standards.
This assumption is likely what has led to the erroneous failure.
Enterprise compute per rack has scaled multiple fold in the last 3-5 years. Alongside the training efficiency gains & datacenter scale increases, even 50T+ is well within reach at the top end.
Yeah, that's insane, you'd need to have many billions of dollars and buy up a huge chunk of the worlds memory supply to do that /s
pretty sure 10 trillion parameters is now the norm among closed ai labs, given that nvidia also references the same 10 trillion number for their nvl72 racks
Pretty bad efficiency then unless that only applies to Fable class but even then - Kimi K3 is around 3T and does similarly well in most benchmarks.
As others have noted, most benchmarks stress the torso, not the tail.
Didn't they say that they can support bigger models now?
Memory capacity on the WSE is the same as before, but access to off-wafer memory is much slower, so the sweet spot is a given fixed balance of memory and compute. They have announced a partnership with AMD in which CPU/GPU hardware is used for part of the workload and the WSE-3 machines are used for inference for specialized smaller models, but I'm not really sure of the details on that.
And there is, of course, the educated guesses about what WSE-4 will be, one being adding a LOT of stacked SRAM or DRAM to tip the balance towards memory (which could also be done by having a few different tile designs with various configurations of compute and memory capacity). I am curious about which way they'll go.
AMD along with cerebras may probably compete with NVIDIA monopoly in near future. Also, NVIDIA will have competition form multiple companies. Just my prediction.
Maybe, but GPU is just one aspect of NVIDIA's dominance. If you are buying Vera Rubin GPUs, you're getting an NVL72 rack, which is only one of several racks that you're probably buying. You'll also need your NVIDIA racks with NVIDIA networking & storage gear, too. At the end of the day, they're "vertically integrated" for your accelerated computing data center (e.g. the "AI Factory"). This doesn't even count the software layer, where CUDA + CUDA-X (not to mention the software for all the sysadmin pieces) has a huge first mover advantage over anyone else.
Is CUDA still a moat? Are we not at the point where frontier models can reimplement software stacks, given you throw enough tokens at the problem.
If you are making a decision to spend 50B on hardware, would you use the proven tech stack or rely on engineers taking an unspecified amount of time vibecoding your software stack while the hardware sits idle?
How about in a month or so when you have to run a slightly different workload?
Hyper scalers like Google or Microsoft, which are the big spenders, have all the incentives in the world to get more out of their gargantuan spending.
In fact both of them, actually Amazon too, invest in their own inference hardware and owns the stack.
You can't possibly think that these companies will keep shelling 50-100B per year in hardware alone where 60%+ is margin for Nvidia and not invest there.
if you are developing your own hardware, you provide your own stack to avoid lawsuits with nvidia. i don't think it's a technical problem at all, but a legal one. this is probably why zluda was scrapped by AMD and Intel. Nvidia technically bans the creation of CUDA reimplementations in their TOS if i remember correctly
Doesn't Google v Oracle provide protection here? Copying APIs is fair use.
If it's patents that are the problem then presumably all these large semiconductor companies have defensive parent portfolios.
I don't know then, amd and Intel decided to get rid of Zluda before that lawsuit ruled that apis are fair use. And now, their cloud customers are ok with using their own stack such as rocm, and Intel got rid of their Garuda ai chips. Those things also happened before the lawsuit was settled
It would be a breach of contract not a copyright issue.
[flagged]
You still need experts to know what is good and what is not.
You just proved that AI cannot currently do that
It can definitely create a software stack for you if you hold it right, but the software stack supported by a trillion dollar company with decades of expertise, that also uses AI to improve its stack is probably gonna be better.
On the reverse side, there are fundamental limits to the number of ways you can perform certain actions, and agents are both diligent as well as able to swarm. If you have your tests beforehand, there is a chance.
i haven't seen it. see the recent browser attempts.
Like NVIDIA bought Groq, AMD might do well buying Cerebras.
AMD did enter into an agreement to buy Taalas, which is speculated [1] will be used to augment their Helios offering.
1. https://www.youtube.com/watch?v=3MKRjt59hh4&pp=0gcJCRMMAYcqI...
They are Ex AMD employees. Nvidia tried back, but they rejected.
The rejection might be overrulled by investors if the offer is enticing enough.
It's not really a far fetched prediction: high margins and huge market attract competition, that's just the law of economics.
Press doubt. Single GPU? Maybe. MultiGPU behemoths like NVL144 and NVL576? I don't think so.
NVLink is at gen9. they had a lot of teething problems and can codesign the hardware and software.
in the name of openness (AMD's only """weapon"""), the UALink spec is a hodgepodge of corporate opinions with very different implementations (looking at you, Broadcom). at spec version 1 (in hardware).
I wish them good luck as I really like AMD, but they compete no more on this than Lambo vs Bugatti.
I was hoping to see Cerebras launch something other than GPT-OSS-120b in production this week, especially with GLM4.7 going away.
If they could launch Qwen 27b or Deepseek Flash that would be amazing.
> CS-4 delivers more than 1,000 tokens per second on models exceeding 10 trillion parameters
Oops did they just out GPT-5.6 sol’s parameter count?
Sol is supposed to be 5T according to rumour. The imminent Astra is allegedly 10
Rumors and allegations aren't worth much. Why don't they just tell us mere mortals?
Because that would reveal their edge to investors, or the lack thereof.
If Fable turns out to be a 10T or 20T model, there is little to boast vs Kimi at 3T. But the opposite is true: if Fable were to be e.g. a 500B model, that would show how far ahead they are from the open models. This isn't likely to be the case ...
I guess there's also the economic aspect. It would make it much easier for competitors to figure out your costs and margins if they know the model parameter sizes you operate.
Why would they? What the upside, for them?
Yeah, its not like this js some sort of Open AI company. That'd be ridiculous.
You think their name means releasing competitive details would be good for them?
I’ve got sone bad news about Federal Express.
Indeed, what is the upside of transparency?
For Linux? Assuring stakeholders that they can both control and observe development and how it works.
For a charity? Assuring donors that funds are being managed appropriately.
For Anthropic? No benefit at all.
Turns out "transparency" is like "weight" or "velocity" in that it has no intrinsic value, and can be positive or negative depending on context.
I mean we kinda know the frontier models are multi trillion parameter models. The only open weights that are close to the frontier are that size too
save qwen3.8 27B which is outclassing much larger models and is in spitting distance of the top 10 in https://artificialanalysis.ai/models#intelligence
I wonder why they removed DeepSWE from their incorporates evaluations
They didn't afaict https://artificialanalysis.ai/agents/coding-agents?coding-ag...
It seems it takes some time to run a new model on all the benchies, not sure they run all models on all of them either
cerebras model are different size then the original models
Just a reminder for everyone that we are only several years and 3 or 4 iterations into hardware being optimized for LLMs. We should all expect orders of magnitude improvement in speed and/or cost over the next 5 years. Then we can have fun conversations about "unlimited" "intelligence" and about what the price wars and profit margins of consumer AI products are when your average ChatGPT user costs the company $0.10 per month.
> CS-4 delivers more than 1,000 tokens per second on models exceeding 10 trillion parameters
Wow!
And, the software side isn't finished being optimized, either. We've seen with Qwen 3.8 27B and DeepSeek V4 Flash 0731 and GLM 5.3 that quite small models can pack a punch. Intelligence density will improve, efficiency of kernels will improve, efficiency of KV caching and MTP will improve, algorithms for splitting workloads across compute units will improve.
It'll all be as cheap as DeepSeek was before the price hike. And, it'll become more and more realistic to run near-frontier intelligence on personal devices.
Hence why taalas was one of the best strategic acquisitions of the year.
I'm honestly baffled they were not acquired by somebody else (sorry AMD).
Taalas will be one of the great disaster investments of the early AI era. It'll be a near total write-down.
The absolute worst market time to etch a model to a chip is right now (very rapid iteration). There is no scenario where they can keep up. The Taalas approach will be viewed as comically foolish within just a few years.
Cerebras will win in terms of approach.
It's 1998: hey, I can drastically speed up your web service, let's etch it right to silicon.
I would still pay ~500 for a chip that runs 10kt/s of a ~100b model on my machine even if the half life is 6months. I m sure my employer would too.
Qwwen3.5 122b was released 6 months ago and is still best in class overall 100-140 B param model.
> I would still pay ~500 for a chip that runs 10kt/s of a ~100b model on my machine even if the half life is 6months.
....
:T
Considering the 8B model uses 53 billion transistors, that's 6.625 transistors per parameter.
https://taalas.com/products/
Assuming they can get it down to 3 (somehow), that's still 300 transistors, or 5.565 RX 9070s.
https://www.techpowerup.com/gpu-specs/radeon-rx-9070.c4250
You're looking at
1) waiting for another 3-5 generations of transistor improvements before it can fit into a single conventional chip, or
2) another generation before getting a monster of a chip (1000+ mm^2), and prices for flawless etching scale quadraticly (likely $1000+ for manufacturing costs alone).
Could happen, but it's a long shot for a market that could be satiated by specialized accelerators.
In Taalas HC2 a chip embeds 20b parameters, and the declared idea is linking the chips. A card with two of them chips and you can already have a dense Qwen at staggering speeds.
Chiplet-style layouts could cut the etching quality requirements per chip down, but it still can't avoid the base cost for manufacturing silicon.
> it still can't avoid the base cost for manufacturing silicon
What costs are you talking about and why would them be a problem?
If it is the price: «Kharya says it costs 100x as much to train a model then to get a customize HC chip in reasonable volumes from Taalas» ( https://www.nextplatform.com/compute/2026/02/19/taalas-etche... ).
That is thinking about an LLM (logical) producer and server. For mass production, the costs go down. And in the case of a ~100b model as the poster mentioned, they would be just single PCI cards with 5 or 6 HC2 chips: doable and practical.
I see more potential problems in the positioning of the SRAM - but not a real problem given that excellent team.
To get a proper idea of the costs the architecture of the HC2 will have to be clearer.
500 what? You’re missing the unit
10000%
> It's 1998: hey, I can drastically speed up your web service, let's etch it right to silicon.
I distinctly remember 32-bit/33 MHz PCI accelerator cards for SSL being a real thing (for use on OpenBSD or FreeBSD), in an era when something like a single core 700 MHz Pentium 3 1U system was a relatively powerful individual bare metal httpd box.
http://www.aster.si/partnerji/compaq/atalla/axl200.html
The CPU load of doing a lot of SSL purely in software was a problem in terms of scaling things up, so this was one attempt at a (very short lived) solution. Note that this predated TLS1.0.
> The absolute worst market time to etch a model to a chip is right now
Slightly disagree. It really depends on the price-point at which they can do that etching. ~1k usd / ~30B model in a hdd-sized case that fits on your desk? I'd buy one right now, even knowing that I'm "stuck" with whatever model of the day is.
Time to market also matters a ton. If they can start shipping chips <1 month after the weights drop that's much more compelling than if it's a 6+ month development pipeline.
In the case of Taalas, the pipeline was said to be 2 months:
> From the moment a previously unseen model is received, it can be realized in hardware in only two months ( https://taalas.com/the-path-to-ubiquitous-ai/ )
It's 2026: let's etch nginx into silicon and get 10,000,000 rps at a cost of 0.1 US/day.
Yes, please!
Most people probably don't care about nginx performance. It shouldn't be your bottleneck unless you serve massive amounts of static data.
In the case of needs to process natural language, instead, massive efficiency (esp. time) can be a game changer. It's like "you have two years to complete the project" vs "you have two hours to complete the project": if you can squeeze that "two years worth" into a negligible delay, it's a game changer.
Ok, how about postgres?
The 500x efficiency gain makes their approach a no brainer. Just make a new chip every 6 months, you still win.
Yes, power efficiency may be the largest benefit of these chips actually - especially if the projections are true that the US and other countries simply aren't able to ramp up power generation to meet forecasted datacenter demand.
There's a bit of a ticking time bomb there, something that a Taalas-like architecture can clearly resolve.
maybe AMD wants the IP to deploy it once ai model development slows down in a few years. Or, their large cloud customers do want to burn through silicon, basically paying rent to AMD for models etched on silicon.
I just want to but hardware so I can run a model at home that is fast. I don't see myself installing a server that burns almost two hundred kilowatts but maybe a card which runs a 27B Qwen...
At 250w when it's working (I understand), and it works for tiny amounts of time per query...
Corrige: a Taalas HC1 PCIe card consumes actually between ~200W and ~250W (a 10 blades server consumes ~2500W).
Etched model into a chip? A… mobile chip eventually? Seems prescient.
> Then we can have fun conversations about "unlimited" "intelligence" and about what the price wars and profit margins of consumer AI products are when your average ChatGPT user costs the company $0.10 per month.
We can have that discussion now: sounds like that would kill OpenAI and Anthropic
A design that bakes the architecture into silicon would be 10x faster, and imagine a version that does all the multiplication ops using single log-amp addition versus dozens of transistors to cut down the amount of silicon used by 50x. The ceiling for AI optimized hardware is extremely high.
Stack on top of that the fact that diffusion based models like the ones made by Inception Labs are far faster and more efficient than autoregressive LLMs and have an even higher ceiling of optimization (single step path prediction via model distillation versus 50 step denoise is currently an active area for image diffusion)
The human brain is soon neither going to be more powerful nor energy efficient than the stuff we use to run AI.
This is part of why I think the data center build-out is a bubble. We've barely scratched the surface when it comes to hardware optimization. We'll see exponential improvements in energy efficiency and speed over the next decade. Exponential, not linear.
GPUs really aren't that great for AI. They just happen to be the best chips we have in mass production right now for this work load, and it takes time to field new designs. Basically every chip engineer on the planet is working on this right now.
Whether it's a bubble or not depends on how much the demand for compute and the type of workload keeps growing, though.
If AI tends to be something used mainly in ideation and development, which is how a lot of people use it today, then once consumer hardware gets good enough you could see a bunch of the current data centre workloads move onto consumer devices.
But if AI starts being used more in repeatable, operational workloads I think it makes sense to have significant cloud infrastructure for it. TBH I haven't seen much of this, and I've been skeptical about people using agents for much of anything when it can be done with just software. But we are starting to see more of this kind of workload, like the taggable Claude in your slack etc that people seem to really love.
By the way, this is the same argument that Michael Burry used to short Nvidia.
He claims that GPU depreciation/obsoletion is much faster than hyperscalers are assuming because new chips will be much better. He's being proved wrong right now because H200 rental prices have been claiming for the last 8 month despite B200 having 10-20x better inference efficiency.[0]
The logic is fundamentally flawed in my opinion. Let's use future Nvidia chips being much better optimized for LLMs for example.
New Nvidia chips 10x better than H200 --> data centers buy a lot --> Nvidia profits a lot.
New Nvidia chips 10x better than H200 --> data centers don't buy --> no faster than expected obsoletion.
In other words, the very act of buying many new Nvidia GPUs would be the event that causes faster than expected obsoletion. Yet, if you don't buy those new Nvidia GPUs, then there is no faster than expected obsoletion.
We also live in a world where there is competition. If Amazon doesn't buy but Microsoft does, suddenly Microsoft can offer better $/token prices.
[0]https://inferencex.semianalysis.com/inference
1. The same isn’t necessarily true of the rest of the hardware stack which may be reused between accelerator generations.
2. You’re missing the “New Nvidia chips 10x B200, compute requirement grows less than 10*software improvements YoY -> buy less Nvidia.” Valuations are based on forward projections (>1T annual for NVDA) which can be revised down leading to a drop in valuation.
> If Amazon doesn't buy but Microsoft does
The big 3 all have their own proprietary accelerators. Meta is buying TPUs as well for now.
I would bet Nvidia’s major customers in 2 years are neoclouds and it seems that Jensen is making the same bet.
1. So this makes Burry’s argument even less convincing since those auxiliary hardware can last longer.
2. Jevons Paradox. More efficiency should lead to bigger models, faster inference, and more total tokens.
3. By all accounts, Trainium and Maia and Meta’s internal chip are struggling to keep up with Nvidia. That’s why they order as many Nvidia chips as possible. They’re not giving up but it isn’t as easy as buying stock Arm cores and taking them to TSMC.
Neoclouds may very well be Nvidia’s biggest customers and this probably what Nvidia wants.
1. Not really, current valuations are priced for persistent 80%+ margins based on spot. If auxiliary hardware lasts longer (I.e. next gen GPU reusing the same shell) then that reduces supply pressure and spot prices.
2. Jevon’s paradox is about total consumption, not margins. Valuations are about margins (and their projections). Many coal mine owners went bust despite increased total coal consumption.
3. Source? Gemini for example is 70% on TPU. I have yet to see data on Maia-300 beyond Microsoft PR. Remember it doesn’t have to be better it has to be more cost efficient. The overwhelming majority of inference spend does not care if token output is 20% slower if it is 50% cheaper.
> Neoclouds may very well be Nvidia’s biggest customers and this probably what Nvidia wants.
What Nvidia needs. Whether neoclouds can stay competitive vs hyperscalers paying Nvidia tax is far from clear, particularly when inference margins compress.
1. The whole Burry argument is that AI hardware becomes obsolete faster. If aux hardware can be reused, that works against the argument.
2. Total consumption drives more demand for the already supply constrained hardware. Can AI hardware market go bust? Sure it can. But being early is the same as being wrong in the investment market. When do you predict the bust to be?
3. Google, Amazon, Microsoft, Meta are all buying as many Nvidia GPUs as they possibly can. The biggest tell on how Nvidia is doing is that their share in inference has increased despite the increase in competition: https://archive.md/CKP0N. So while competition is getting bigger and bigger because the overall pie is getting exponentially bigger, Nvidia's growth is still higher than average.
Some other sources:
https://www.businessinsider.com/amazon-nvidia-aws-ai-chip-do...
https://www.businessinsider.com/startups-amazon-ai-chips-les...
> 1. The whole Burry argument is that AI hardware becomes obsolete faster. If aux hardware can be reused, that works against the argument.
Burry’s main argument is depreciation is being understated and the capex vintages will not be paid off before they are essentially useless. This can happen whether or not aux is reused.
> Total consumption drives more demand for the already supply constrained hardware.
Demand is the wrong metric.
Only number that matters is whether AI attributable revenue will be sufficient to pay back enough of each successive capex vintage (e.g. 750B this year, 1T next year, 1.2T in 2028) so that hyperscalers and neoclouds can either self-fund or continue to issue debt as bond markets are already straining and tax-payer backed sovereign debt is providing a high baseline. Otherwise they downgrade capex projections and the bubble pops.
Expensive compute needs expensive inference to justify 30-40B/year/GW of compute. There are many reasons why frontier API pricing which is what the industry is based on may not persist. It is also almost certainly the case that 2026 is the worst year of supply and demand mismatch to allow for 80%+ margins. HBF next year has the potential to single handedly pop the DRAM spot bubble.
> Can AI hardware market go bust? Sure it can.
This is the bear thesis. It is not that AI will crash or be useless.
> But being early is the same as being wrong in the investment market. When do you predict the bust to be?
Q4 27-Q2 28 is when the bill becomes due at the latest. There are sufficient financial levers left to buy time without returns until then.
> Google, Amazon, Microsoft, Meta are all buying as many Nvidia GPUs as they possibly can.
All of these companies have rock solid revenue streams and can easily swallow 500B of capex devaluation over time. Their buying of Nvidia today is not necessarily the indicator you are implying as there are strong competitive reasons to make the game more expensive for everyone else.
The rest of your argument centers around whether AI growth will meet the cap ex expenses. I don't see anything new in it.
I'll believe it when I see it. Jevons paradox will apply here again in my opinion. HBF does not replace HBM.I wonder: in world where inference is cheap, how many engineering agents that use simulation as their feedback we will use?
In the scenario, engineering everything becomes so easy - so why not optimize everything? every component, every product, every system?
And maybe llm's could invent. So even more to simulate. And simulation is inherently compute-heavy.
So unless there are some other bottlenecks, we'll use a lot of simulation servers.
On the plus side, lots of cheap servers to swoop up :)
But power hungry.
In that 5+ year timeline, the compute per watt could change by three orders of magnitude.
GPUs are to LLMs what CPUs are to gaming — not a good fit.
A cursory estimate courtesy of ChatGPT suggests that there is a grand total of one order of magnitude or less of power efficiency improvement available compared to current Blackwell if the entire system’s power consumption outside the ALUs went all the way to zero.
If you want three orders of magnitude improvement, you probably need to find two of those orders of magnitude somewhere else: process improvements, different ALU design, model architecture changes, etc.
Oh, you could go analog rather than digital. Ever look at ALU design? Nothing efficient about it!
Look at their power supply, it’s not something you can run in a home lab. Unfortunately most of that will likely go to the bin eventually :(
No but dedicated server prices will likely drop or at least you'll get more for your money
By the time these gigawatt datacenters are done being built the hardware will be so far behind state of the art they may be mostly useless.
True, I have to agree with you. The AI giants might be investing a huge amount of money in generation 1 technology. There might be a much better way to do it just around the corner. They might know this and thus the hurry to IPO.
A rough analogy would be if the first generation of ISP's spent billions on dial-up exchanges, when fibre could be invented next year.
> We'll see exponential improvements in energy efficiency and speed over the next decade. Exponential, not linear.
Why? Make your case.
Exactly right, and nVidia is protecting their moat through business practices rather than genuine product innovation.
What LLM-specific hardware improvements should one expect? Seems to me that LLM inference is simple architecturally (matmul et al) so most scaling in hardware should come from general improvements (memory BW, packaging, interconnect, power).
What you describe is basically Cerebras case, at the bottom it's just a really big die (about x28 an NVIDIA GB200) with a lot of work to reduce memory latency and improve throughput. What it's actually amazing is how can they make a chip so big and still have a decent yield to be commercially viable.
Congratulations! You have just realized that the AI data center build out is a total scam, built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling.
There exist other AI accelerators (TPUs, ASICs) that perfectly exceed the throughput that LLMs need to scale as well. But the true solution is more software optimizations. There's a tiny handful of them but more needs to be discovered so that we can reduce building hundreds of more data centers as the alternatives mature.
As better software becomes more useful for the alternative AI hardware for developers with LLMs running efficiently you then would have more choices of hardware to run your LLMs on rather than just only GPUs.
TPUs and ASICs run in data centers too. Your argument only holds true if there's some satisfied limit to demand for inference. If not, data centers will continue to spring up to host more and more agents. Even if agents were running on hardware and software as efficient as the human brain, its conceivable we want trillions of them running at any given time which would require data center scale.
Everything has some satisfied limit to demand, often depending on the price. If you assume there will never be any satisfied limit to demand for inference at any price you can justify any investment.
Looking back nearly 80 years, what has been the limit to transistor demand so far?
Unlimited.
What has been the limit to electricity demand globally?
Unlimited.
We can't get enough and never will. Costs have to become pretty severe to turn back the demand as well.
Yeah, but there's certainly a part of the curve where price drops by X OOMs and demand increases by much more than X OOMs. (Presumably some of that is substitution and some of that is new use cases.)
So far, at least by Openrouter's weekly numbers, there doesn't seem to be a satisfied limit.
https://openrouter.ai/rankings#top-models
And their market share sits at around 16-20%.
At 75.3 trillion tokens for the week ending 10 Aug 2026, that means that up to 450 trillion tokens were plausibly demanded by the whole market for that week.
My take: At max saturation, each person on earth could have their demands satiated by an average of 16 agents running concurrently. Sometimes more, often times less, but the average would likely be at 16.
At 200 tokens/second for each agent, that would mean 15.48288 quintillion tokens per week.
We're currently at about 0.00290643601% of the calculated demand ceiling.
Even if the demand limit per person is just 1 agent at 50 tokens/second, the current demand's still 0.186011905% of the theoretical ceiling.
Is this article not exactly about ASICs built for LLM inference and training acceleration?
I wonder what this looks like in 5 years... Will there be a massive push to repurpose these giant boxes into housing? Will they get turned back into the farm land from where they came? When a data center goes bust, what happens to the parts left behind?
I'd think the infrastructure would tend towards factories, smelters, and so on. Industrial things that have reasonably high power demands, can use the building, and don't care about the lack of windows.
They're typically not built where you want housing, and the buildings are distinctly the wrong shape.
If you can't use the power infrastructure profitably my next thought would be warehousing.
But also... we've seen a pretty continually increasing demand for compute. Even if AI busts a bit (or becomes a bit more efficient) I bet most data centres stay data centres, just less profitable ones.
Huh, why I'm not surprised that HN is full of opinions confidently stated without any numbers or resources to back up?
> built on both the insurmountable trillions of debt, and the assumption that only GPUs are all we need to continue scaling.
Insurmountable according to whom? And who assume that only GPUs are all we need to continue scaling? Google, Amazon, Microsoft, Meta and OpenAI, all have or plan custom non-GPU AI chips. Do they plan to use them not for scaling?
KV caching status?
What's the point of 1000tok/s if you have to do prefill on every agentic turn which at 100k depth would make it 1.5 min latency every turn?
Information about RAM type/size and connection topology of the RAM to be used for context cache seems to be conspicuously absent from the slick looking marketing materials.
There's a few more details at the bottom of this page: https://www.cerebras.ai/blog/introducing-cerebras-cs-4
44GB on-chip-sram * 3 chips. Per chip: 43.2 PB/s memory access + 53.5 PB/s on-chip fabric bandwidth + 2.4 Tbits/s "IO" bandwidth (I think that means their RoCE v2 RDMA over Ethernet interface).
I suspect there might be a certain amount of customization for how much RAM they attach when you order it.
They have managed to make the external link 300GBps/2us. Cs3 was 150/5.
This is 1/3rd blackwells nvlink c2c bandwidth already. Not too bad. We can make KV cache offload work with that I suppose.
If magically KV cache was not an issue, pipeline parallelism on cerebras can be quite pleasant. As for the KV cache offload, I have hopes their CPO solution they're trying with that canadian company ends up bearing fruit.
If cerebars is performing well, why didn't its predecessor, server S-3, become the largest API token provider on OpenRouter, surpassing the official model releases?
Without having any inside information, one possible theory:
All or a vast majority of of the cerebras manufacturing capacity was going to a few companies that aren't publicly available inference providers on openrouter, for their own internal use.
or
The asking price of the S-3, no matter how speedy it might be, for small/medium size customers made it economically prohibitive to purchase and use to sell public inference vs. buying more common nvidia b200 or whatever.
Cerebras provides high-speed inference at high cost. It's never going to be the cheapest and thus it will probably remain niche.
But that's supply and demand, not technology. Right now a lot more people want their inference than they can supply. as supply catches up in the next 5-10 years, the underlying tech at scale is probably cheaper than GPUs per token produced.
it only takes ~445 GB300 NVL72 (about $22b) to run ALL of openrouter demand for a year. Microsoft rolled out $32b of DC 2026Q1.
imo the issue is that most openrouter demand is inauthentic activity (things that anthropic and openai models will refuse to do like pretend to not be bots when interacting with humans)
I thought your numbers must be wrong.
So I plugged 288 trillion tokens/month (OpenRouter's current rate), 500 billion MoE model average, and the math comes out to be around 620 B200 GPUs minimum.
So basically, OpenRouter's volume must be absolutely tiny compared to the volume hyperscalers are getting.
For reference, Google serves >3 quadrillion/mo.
[1] https://x.com/ren_stocks/status/2056946641815396718?s=20
Which is only 10x more than Open Router by the way.
It is worth mentioning, the OpenRouter demand isn't static though. It has increased week on week since early 2026.
I was curious so I looked it up: looks like a GB300 NVL72 is about $4M. So $22B would buy you 5500 such racks, no?
GPU cost is about half of datacenter cost. Other half is cooling, power and networking.
Probably the same reason why there are more people who takes buses, subways, trains than drive Ferraris.
I love how instead of comparing a Ferrari (fast and expensive) to some average car (not fast, not expensive) to make your point..you went for public transport where your comparison cracks from multiple angles.
Because comparing it to an average car is wrong. It isn't about fast/expensive. It is about fast/capacity.
A Ferrari can seat up to 4 people[0], which is about the same as the average car. Capacity doesn't change much.
Meanwhile, a bus/subway system is meant to support millions of people. Tokyo's metro has to support up to 37 million people. You can't do that with Ferraris.
[0]https://www.ferrari.com/en-EN/auto/ferrari-purosangue
If you're willing to pay a significant premium for latency, why use openrouter? And anyway Cerebras only supported a few specific models.
The WSE is very expensive to build, and they have a waiting list of customers who are already willing to pay a lot of money for the available supply.
Cerebras capacity was pretty much entirely bought out at some point. We needed it and couldn't get it.
Because they aren't selling inference, they're selling hardware. The only reason they sell any tokens on OpenRouter is so they get on the benchmark that shows them as the fastest provider. It's free advertising.
I use Cerebras via OpenRouter. It’s every bit as fast and reliable for my needs as claimed. I suspect the reason is that they can either be making peanuts selling inference to plebs like me via OpenRouter, or making bank selling the more expensive models to businesses directly. In short: I would be very surprised if they have die capacity, and are at this point maximising revenue per chip.
Cerebras is very fast but you can basically never use it because of its scarcity
Conspicuously missing: power consumption figures
162 kW
I guess we know why there's a fair bit of investment money going into small modular nuclear reactor startups now.
And advanced geothermal. Fervo Energy let's us get energy that's not based on burning fossil fuels but is, instead, able to produce energy from the ground.
Remove energy from the ground. I wonder what the consequences may be once we are cooling the underground at several MW/h.
Earth will be swallowed by the sun before humans could make a dent in the earths core temperature.
The core is obviously not in question here, but at scale and long term it could have an impact on the water bed, vegetation, underground ecosystems, soil stability, etc.
Actually that's mostly just the military funding those, with a few of them having data center partnerships so they can shield themselves from the criticism of what they really are: military contractors.
I heard a great deal of noise from early 2002 to the present date that the US military had a high interest in small portable nuclear reactors for large bases in Iraq and Afghanistan. And particularly around the peak period of troops on the ground in AF and IQ. And indeed a place like Bagram or Kandahar used a shitton of diesel to run generators. But nothing ever came to fruition to actually implement it, has something changed now that they actually consider it worth doing?
God, I was going to ask if this could be deployed in a standard existing datacenter, but I guess that answers that question.
So the answer is yes? Putting in a few of those racks for special tasks shouldn't break the power assumptions of a data center.
Distribution's always the problem - how much power actually gets delivered to each rack. 162 is ~an order of magnitude higher than normal, which means nothing in a standard data center is going to be built to deliver that kind of power to one rack.
I presume per rack?
Can you imagine something radiating that much energy into a space in your home?
It's mandatory liquid cooling, so it's meant to be attached to a specialized liquid cooling loop that gets the heat outside the building.
This is far beyond the practical maximums of like 10 to 15kW per 44U cabinet front to rear air cooling for 'regular' rackmount server stuff.
Indeed. You need 45 to 60 liters per second of cooling water flowing over a Cerebras wafer every minute to keep it under 90C. And that’s assuming the water leaves at 90C…
More realistically, you need much more cooling water.
CS-3 used 100 liters of water with a cold plate (https://www.brownstoneresearch.com/bleeding-edge/ai-infrastr...). But you can also use refrigerants with a cold plate (https://eng.umd.edu/engineering-ai-public-good/cooling-data-...) or dielectric fluid in total immersion. Until they find a more power-efficient design, my guess is total immersion will come back in style.
I guess because I have actually set foot in a data center I don't imagine literally every product in my home.
"10x more throughput per watt than CS-3"
Impressive that this is an "interim" product, the start of a new line that ought to be continued with the WSE-4 family, where they are supposed to use a 3nm process and, maybe, 3D stacked SRAM. The modular architecture also points towards field upgrades that are badly needed for AI datacenter builders.
Interestingly they’re still on the WSE-3 (5nm TSMC) wafer chip and slightly bumped up the specs there (overlocking mostly it seems), for why it’s called WSE-3 Turbo now. I think people were also expecting WSE-4, as it’s been 2 years now since WSE-3 was launched.
It would be even better if a version available to individual users were released soon.
They do offer API services to individual users... though with a set of models that makes it unlikely that you want to use it. They are promising Qwen 3.8 27B any day now though*.
if you have the money as an "individual user" to purchase one of their racks... save your money and retire.
* Actually they sent out an email claiming they already have it, but I don't seem to have access, they're promising to release it to the "shared tier" any day now.
> save your money and retire.
Now that this hypothetical person has retired, what are they gonna do all day? Just sit on the beach and drink Mai Tais? If that's what they wanna do, sure, but nerds gonna nerd, and if I had that kind of money to retire on, I'd totally buy some ridiculously expensive AI box for fun.
Ah but if you have the kind of money where this is a reasonable retirement hobby purchase, you aren't bothered by representing yourself as a "enterprise" :P
not only that, but I was so happy with their GLM 4.8 that they got rid of yesterday :(
What do they do with old ones? Their hardware physically can't run other models right?
The Cerebras hardware is not locked to specific models / model families. Taalas is the company that's etching models into their silicon, locking it to that model forever.
That's pretty sweet. Thanks
I’ll get that 250kW home power service dropped in next week!
For now you can rig an adapter to your nearest DC EV charging station, but make sure it's near a body of water for the cooling.
Hot water for the whole neighbourhood!
I'd like to see a consumer version too, I don't need a whole rack of them. I probably can't even afford one gpu-sized one
needs an sla that says power will never ever ever go out or else you will have a useless shattered plate of silicon.
Huh, why would it shatter if the power goes out?
by my understanding:
there is an extensive and complicated cooling system that permeates the wafer. some of the cores are completely turned off because they fail qc (see the tsmc logo) if all of their neighbors have been going at full bore the thermal differential can cause stress fractures if the cooling system suddenly fails.
can these vibe coded sites please set a max width and overflow so their sites work fine on mobile
Good news, future models will have your comment in their training set, making them slightly more likely to fix that problem!
What a time to be alive
Five years from now, I don't know why anyone will still be using Nvidia for inference. Note that Cerebras is for inference only, not for training.
I understand that Cerebras has competition, but this bodes even more poorly for Nvidia for inference. Nvidia may still have a role to play for training, however.
NVIDIA has the best supply chain in the entire game. They are the only ones who can produce at their scale. You really shouldn’t underestimate their position
Nvidia is at this time a pretty well run company tech wise. They are going to keep iterating on the inferencing hardware stack over the next five years too.
The only way I see in which Nvidia can catch up is by buying Cerebras.
Cerebras is only claiming ~2x the performance of Groqvidia which usually isn't enough for people to switch.
What's the sticker price? If I have 20 million in the bank can I just like buy one or what
Surely it would depend on your model, since they need to etch this into silicon.
You're thinking of Taalas.
OpenAI needs to immediately move to acquire Cerebras.
Nvidia's extreme margin is the opportunity for OpenAI's cost reduction. Buying Cerebras would pay for itself and they should take all of its future production (after filling required contracts).
Right now China's models have no silicon moat. Cerebras as a drastic speed-up / cost-reduction potential, can assist in building a competitive moat. And every time a Cerebras pops up, OpenAI or Anthropic should eat them if at all possible.
There's no stand-alone frontier AI company of great scale in the near future that doesn't have a large silicon advantage in-house. Apple knew it in smartphones, Google figured it out a long time ago as well.
Do you know about Jalapeno?
^
OpenAI is partnering with Cerebras while simultaneously investing in their own silicon play. Hedged bets.
After sitting thru their keynote today, it makes sense. The main throughput speedups they tout are an obvious evolution of the GPU that all companies will be building in the next year. Wafer-scale interconnected memory and compute is just going to beat out mountains of network cabling any day on both cost and performance metrics.
With what? More debt? What will nvidia say?
> Introducing the all new Cerebras CS-4, a revolutionary rack-scale solution that delivers upto 30x faster inference compared to GPUs, enhanced economics, and a simple path todeploy [sic] hyperscale capacity.
Did nobody proofread this?
If they had ask Claude it would probably look like this: Introducing the all new Cerebras CS-4, a revolutionary rack-scale solution that delivers up to 30x faster inference compared to GPUs, enhanced economics, and a simple path to load-bearing hyper scale capacity.
That's unusually honest and the sharpest thing in this thread.
You're underselling it, and here's why.
Sometimes I wonder if mistakes are now used to indicate the possibility that a human actually wrote it.
There’s been a spate of Reddit AI bots using all lower case in hopes of evading detection.
It’s still incredibly obvious.
I don’t really visit Reddit much these days but would love to see an example.
https://www.reddit.com/r/ModSupport/comments/1tu67kn/influx_...
Maybe it is just part of their "compact design".
Well at lest it's written by a human.
« Make it look like human written »
An error no frontier LLM would make, eh
That's a Lord of the Rings length book from a frontier model (Sol) every 15 minutes.
The comparison seems incomplete. CS‑4 is a full rack-scale system with three wafer-scale processors, but the exact GPU models, GPU count, power consumption, price information are not disclosed. We still don't know if buying a multi-GPU rack (or racks) is cheaper and/or more efficient in power. The fact that they didn't disclose these numbers makes me believe that the numbers are not in their favor. And personally, makes me see them as disingenuous.
> enabling massive clusters and models with more than 50 trillion parameters
Really wish they’d host more models for us normies. My guess is OAI will buy / subsidize them with terms that will close off open models.
I wonder what are the benchmarks of hashcat on different hashes.
That "GPU" comparison is the vaguest i seen so far
True, it's also "per user", somehow, but I think it's a misleading metric. Cerebras chips take the whole wafer?
A single TSMC wafer contains 60 to 65 B200s, assuming 70% yields that's 40ish wafers per die.
Cerebras cannot redefine wafer economics.
Depends on what you mean, they have more redundancy which means that the yield can be much higher.
only 44GB * 3 of VRAM per rack :O
I guess you'd need a DOZEN(s) of these to host a large model with long context KV caches?
Cerebras should slowly also move to dgx/ryzen market for a desktop version for masses at affordable price yet providing substantial tokens/second on desktop
Desktop SRAM isn't really viable because it could cost $100K just to load the model.
...so, in the same ballapark as ddr5? :-)
Is it just me or is it bizarre that they're advertising old open-weight models.
GLM 4.7 (December 2025) not 5 (Feb) 5.1 (April) or 5.2 (June). 5.3 (4 days ago) is, to be fair, not open weights yet... but there's a lot since 4.7.
Kimi K2.7 (April) not K2.7-code (June) or K3 (July).
Gemma 4 (April), Llama (April), and gpt-oss (August 2025) are up to date, but old (for models).
Meanwhile the closed source GPT 5.6 sol is up to date (June)...
Should potential purchasers take away from this that they're not going to be able to run recent models unless they front the cost of developing software or something?
I think they run whatever models they get paid to run. But mostly from enterprise. They are clearly not interested in consumer dollars.
I mean the product is a server rack and while there's no advertised price I would assume it's six figures. So yes, an enterprise product.
But even an enterprise is going to care about the difference between "we can run the model we want with support from the manufacturer" and "we have to purchase the product, and then spend another 6 figure sum having developers port a recent model to the product to use it".
You’re at least an order or magnitude under… likely two.
A single AI server with a mere 8 GPUs from Nvidia is already mid 6 digits. A rack system from Nvidia is mid 7 digits.
There’s some info out there that suggests the CS1 had an 8 digits price tag, so it wouldn’t be surprising to see that here.
Yeah, did some googling after WarmWash's comment and I concur.
I feel like 6-figures would be the clearance price on it...
6 figures is a single mid range Xeon or Epyc server these days.
this is cool!
[dead]