I haven't kept up with the latest on supercomputing power, but I recall some years ago there being strong evidence that China had a couple of un-announced supercomputers that would have topped the charts. It makes me wonder what is publicly disclosed vs. actual.
> Why aren’t these AI companies submitting to the TOP500 to show off their computing prowess?
my knowledge is 10+ years out of date, but once upon a time if they'd chosen to, Google could have had _several_ entries in the top 10 of the TOP500 list
It's just poker, they didn't want to tip their hand
I’ve worked on several systems that had enough flop/s to make it in the top 5-10, but for which we never submitted benchmarks. Sometimes their backend network layout technically would make them several smaller clusters for an HPL run, sometimes it’s because the cluster is too heterogeneous to get a good benchmark result, and sometimes it’s because the employer wants to keep a low profile.
Most of the time, it just that it’s a hassle. It takes a while to prep and tune a big hero run for benchmarking, and if you spend a billion dollars on a cluster, it’s making you a lot more than that. Taking it down for a day or two stops the money printers.
Why would the scientific computing people want to tip their hand? It’s an open secret that the main point of these mammoth FP64 compute machines is to simulate nuclear weapons detonations to comply with the CTBT you’d think that crowd would really not be fans of broadcasting their capabilities.
Even before AI training clusters became important, Google has had an outstanding custom fabric (there's papers about it) together with the ability to tune NICs for their own cases, and "their own cases" meant nearly everything engineered within Google. Ethernet hardware has had low kernel latency and DMA for a long time; it's the rest of the stack that hurts. But as far back as the early 2010s (if not further back, that goes beyond my knowledge horizon), you could just make it not hurt, if you had the software engineers to do it.
Historically there have been a bunch of clusters on the Top 500 that weren't used for HPC. The tell is that they used Ethernet (this was before RoCE). It's less efficient but you can still get an OK Linpack score.
TOP500 hasn't been a particularly useful measure of practical computing power in modern systems for many years because what it measures isn't a significant bottleneck in most real systems. It has become a measure of how much money someone is willing to spend for bragging rights. (HPCG is better in that it is a bit more bandwidth focused but still pretty narrow.)
Most companies with huge systems don't participate.
I wonder if there would have been an opportunity to generate some finer-grained benchmarks with something like BiCGStab+ILU (or maybe CG+incomplete cholesky). Instead of CG+Gauss Seidel. The pitch being, you might have made different memory vs compute trade-offs with designing your cluster, but you should be able to select a fill-in factor for the preconditioner to suit it.
> We think it is highly likely that these LX2 chiplets are etched using SMIC 7 nanometer processes at the N+3 refinement, and we base that on the fact that the chip only runs at 1.55 GHz. That is nowhere near the 3 GHz that SMIC can push with that process, but it is probably lower to get the memory and core speeds more balanced. [1]
Interesting to see PAC mentioned on the slide, I'd have assumed security features would be a waste of transistors on something so compute-optimized - but maybe they want to isolate workloads from each other?
Why are they not using GPUs? is it use cases that don't suit GPUs or because of the limitations they are imposing on themselves to use SMIC domestic chips?
It is likely that those cores are dedicated to unrelated management, monitoring, and administrative tasks. This is common and many workloads are throttled on bandwidth anyway. For the purposes of the benchmark, those cores are not participating in the workload.
And then even if some nodes had 40/40 "good" cores, it would make load balancing a lot more complicated if core counts vary. Easier to turn them off at the hardware level.
TOP500 can be done with inexpensive silicon. It is more about a willingness to aggregate enough hardware in one place. As a benchmark, it tells you almost nothing about computing power or scalability for other applications because it doesn't exercise the bottlenecks most high-scale applications have.
We're too busy regulating the tech, not granting access to US engineers and companies, arguing against power and data centers, stopping skilled immigration.
This is absolutely going to bite us in the face in five to ten years.
Separate issue that has nothing to do with US manufacturing or HPC. I think our retreat from science funding and offshoring advanced manufacturing is a bigger issue.
I haven't kept up with the latest on supercomputing power, but I recall some years ago there being strong evidence that China had a couple of un-announced supercomputers that would have topped the charts. It makes me wonder what is publicly disclosed vs. actual.
> Why aren’t these AI companies submitting to the TOP500 to show off their computing prowess?
my knowledge is 10+ years out of date, but once upon a time if they'd chosen to, Google could have had _several_ entries in the top 10 of the TOP500 list
It's just poker, they didn't want to tip their hand
Also, would those 550k Blackwell have good FP64 performance? How would one even compare them?
I’ve worked on several systems that had enough flop/s to make it in the top 5-10, but for which we never submitted benchmarks. Sometimes their backend network layout technically would make them several smaller clusters for an HPL run, sometimes it’s because the cluster is too heterogeneous to get a good benchmark result, and sometimes it’s because the employer wants to keep a low profile.
Most of the time, it just that it’s a hassle. It takes a while to prep and tune a big hero run for benchmarking, and if you spend a billion dollars on a cluster, it’s making you a lot more than that. Taking it down for a day or two stops the money printers.
Why would the scientific computing people want to tip their hand? It’s an open secret that the main point of these mammoth FP64 compute machines is to simulate nuclear weapons detonations to comply with the CTBT you’d think that crowd would really not be fans of broadcasting their capabilities.
Cloud computing is not a supercomputer. Different architecture, bandwitch, interconnectivity and latencies.
That's not nearly as true when you look at AI training clusters. They're basically supercomputers but without an FP64 focus.
(These are the systems to which GP was referring at Google.)
Even before AI training clusters became important, Google has had an outstanding custom fabric (there's papers about it) together with the ability to tune NICs for their own cases, and "their own cases" meant nearly everything engineered within Google. Ethernet hardware has had low kernel latency and DMA for a long time; it's the rest of the stack that hurts. But as far back as the early 2010s (if not further back, that goes beyond my knowledge horizon), you could just make it not hurt, if you had the software engineers to do it.
I thought TPUs couldn't reasonably run LINPACK at all because TPUs do not acknowledge that FP64 exists.
I know Google wants to compare their stuff to El Capitan or whatever but the comparison does not seem valid to me.
Historically there have been a bunch of clusters on the Top 500 that weren't used for HPC. The tell is that they used Ethernet (this was before RoCE). It's less efficient but you can still get an OK Linpack score.
TOP500 hasn't been a particularly useful measure of practical computing power in modern systems for many years because what it measures isn't a significant bottleneck in most real systems. It has become a measure of how much money someone is willing to spend for bragging rights. (HPCG is better in that it is a bit more bandwidth focused but still pretty narrow.)
Most companies with huge systems don't participate.
I wonder if there would have been an opportunity to generate some finer-grained benchmarks with something like BiCGStab+ILU (or maybe CG+incomplete cholesky). Instead of CG+Gauss Seidel. The pitch being, you might have made different memory vs compute trade-offs with designing your cluster, but you should be able to select a fill-in factor for the preconditioner to suit it.
> We think it is highly likely that these LX2 chiplets are etched using SMIC 7 nanometer processes at the N+3 refinement, and we base that on the fact that the chip only runs at 1.55 GHz. That is nowhere near the 3 GHz that SMIC can push with that process, but it is probably lower to get the memory and core speeds more balanced. [1]
Based on the ARMv9.2.
[1] https://www.nextplatform.com/hpc/2026/06/25/a-deep-dive-on-c...
Interesting to see PAC mentioned on the slide, I'd have assumed security features would be a waste of transistors on something so compute-optimized - but maybe they want to isolate workloads from each other?
Why are they not using GPUs? is it use cases that don't suit GPUs or because of the limitations they are imposing on themselves to use SMIC domestic chips?
GPUs are for graphics (the G in GPU). These systems are used for more general computations.
> Two cores are disabled per cluster.
I’m sure there is a good reason for this, which is..?
It is likely that those cores are dedicated to unrelated management, monitoring, and administrative tasks. This is common and many workloads are throttled on bandwidth anyway. For the purposes of the benchmark, those cores are not participating in the workload.
Yield. Some fraction of cores had a speck of dust or something, but at 38/40 good cores per chip they got economical yield
And then even if some nodes had 40/40 "good" cores, it would make load balancing a lot more complicated if core counts vary. Easier to turn them off at the hardware level.
Couldn't some chips have 40 good cores, while others have only 36? Do they all need to be exactly 38?
Extremely impressive accomplishment considering they did this with Chinese interconnects and Chinese chips. This is a wake up call.
TOP500 can be done with inexpensive silicon. It is more about a willingness to aggregate enough hardware in one place. As a benchmark, it tells you almost nothing about computing power or scalability for other applications because it doesn't exercise the bottlenecks most high-scale applications have.
We're too busy regulating the tech, not granting access to US engineers and companies, arguing against power and data centers, stopping skilled immigration.
This is absolutely going to bite us in the face in five to ten years.
Separate issue that has nothing to do with US manufacturing or HPC. I think our retreat from science funding and offshoring advanced manufacturing is a bigger issue.
Previously on HN: https://news.ycombinator.com/item?id=48658334
Would the AI “GW-scale” clusters be able to run the Top500 benchmarks meaningfully? And what might be the outcome?
Yes, they should score well on Linpack as long as they use Ozaki emulation.
Just glad to see Hamburg mentioned :) Hope you all didn’t suffer too much through the current heatwave
How many tokens/s? :)
Is it the first to reach 2 exaflops?