> run these benchmarks as an integration tests on every single cluster maintenance
Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?
For the same reason that we don't test engines during a race. Only what's under test should change, with the rest being fixed.
It would be much more difficult to adapt some of your own applications as a test. If the result are bad, how would you even know if it's the application's fault or a problem with the cluster? LINPACK on the other hand is well understood and there has been tons of work to make sure that it uses all the power your cluster can deliver.
When a company claims that their new vaccine cures cancer, but is against applying for FDA approval because "no reasons given", you seem to be the kind of person that camps in front of their offices overnight to get the first 10 shots, all in the neck.
Tesla is saying their hardware is 10x better than the competition.
I don't believe it. They don't publish any numbers, and everybody else does.
You believe them. Good for you.
I don't. I think not doing that is extremely suspicious, because if their cluster can be turned on, they have the results. So the reality of this is 100% certain that their results just suck, and they don't publish them to save face.
You believe their shit is so good, they didn't even test it while installing it. Good for you.
You think there is a masterplan to keep the results secret. Good for you.
I think their results are just horrible, and that's why they don't show them.
With any serious company, this wouldn't even be a discussion, because they would just show the verified facts about what their cluster can do or shut up about it if its really so secret.
You don't see the NSA or the DOD bragging around about being 100x better than "the competition".
To me they look forced to sell that they are doing something about these clusters to... drive hype up and get the stocks back up, and since their numbers are actually horrible, this is what we get.
But they've got no obligation to prove anything to random people on the internet. They aren't asking for peer review. They probably don't want to waste time and energy running a benchmark to compete in someone else's top N list. What's in it for them?
> But they've got no obligation to prove anything to random people on the internet.
They are a publicly traded company.
I'm a tesla investor and deserve to know.
If they are right and their hardware is 10x better than the competitor, then I'm happy. If they are wrong and they could have get 10x better outcomes by spending 100x less money, I'll be very pissed off.
> They probably don't want to waste time and energy running a benchmark to compete in someone else's top N list.
It costs no time. It's the first thing anybody does when building any cluster. It's part of accepting the materials from your suppliers to check that they didn't sell you shit.
I'm not an expert in securities law, but I don't believe being an investor entitles you to demand arbitrary proprietary information you want from a company. Otherwise people would buy one Apple share and find out what specs the new iPhone will have.
HPL is a stress test with a notionally useful output measurement - just about the most effective way of pushing CPU load to the limit, and it tells you what fraction of the theoretical maximum FLOPS you can actually achieve given the other system constraints like memory and network throughput.
Why run someone else's benchmark and not your own application to test performance? And what's the point of submitting to Top500? Why do you care how your system ranks? What's the business or technical purpose in that?