Benchmarks deleted geekbench over cheating allegations

BENCHMARKS DELETED GEEKBENCH OVER CHEATING ALLEGATIONS PRO

Naturally, and unfortunately, my first thought was that there must be some sort of cheating going on. The whole thing ended up as quite the trip down the rabbit hole. ItвЂ™s when you see such odd results that alarm bells go off as thereвЂ™s something that is quite amiss. Things got weirder when I received a Chinese Reno3 with the MediaTek Dimensity 1000L вЂ“ a much more powerful and recent chip, but which for some reason performed worse than its P95 sibling. The phone surprised me quite a bit at first, as in systems benchmarks such as PCMark it was punching quite above its weight and what I had expected out of a Cortex-A75 class SoC.

BENCHMARKS DELETED GEEKBENCH OVER CHEATING ALLEGATIONS PRO

The whole thing got to my attention when I had first received OppoвЂ™s new Reno3 Pro вЂ“ the European version with MediaTekвЂ™s Helio P95 chipset. Helio P95 outperforming Dimensity 1000L?! In most past cases we always blamed the device vendors for cheating as it had been their mechanisms and initiative вЂ“ we hadnвЂ™t had evidence of enablement by chipset vendors, at least until now. The one big difference here however is that thereвЂ™s always been somewhat of a firewall in our coverage between what a device vendor did, and what chip vendors enabled them to do, and thatвЂ™s where we come to MediaTekвЂ™s behavior over the last few years. Cheating in benchmarks here continued to be a very real problem and commonplace practice.В HuaweiвЂ™s rationale back then was that they felt that they needed to do it because others did it as well вЂ“ and they didnвЂ™t want to lose face to the competition in regards to the marketing power of benchmark numbers. The problem is, the Chinese vendor market is still huge, and weвЂ™re not able to dissect every single device and vendor out there. The naming and shaming here again helped, as the company had transitioned from employing invisible mechanisms to something that was a lot more honest and transparent, and a lot less problematic for follow-up devices. Most predominantly for our more western audience this happened to Huawei just a couple of generations ago with mechanisms that essentially disabled thermal throttling the of phones вЂ“ letting more demanding benchmarks essentially have the SoC burn through to the maximum until thermal shutdowns. In recent years however we saw a big resurgence of such methods, particularly from Chinese vendors. The naming and shaming did work over the following years, as vendors quickly abandoned such methods out of fear of media backlash вЂ“ the negatives far outweighed the positives. The Samsung case eventually even ended up with a successful $13.4m class-action lawsuit judgment against the company вЂ“ with yours truly and AnandTech even being cited in the court filing. I remember back in 2013 where I had tipped off Brian and Anand about some of the shenanigans Samsung was doing on the GPU of Exynos chipsets on the Galaxy S4, only for the thing to blow up into a wider analysis of the practice amongst many of the mobile vendors back then вЂ“ with all of them being foundВ guilty.

Mobile benchmark cheating hasВ a long story that goes far back for the industry (well вЂ“ at least in smartphone industry years), and has also been a controversial coverage topic at AnandTech for several years now.

Benchmarks deleted geekbench over cheating allegations