We always say that benchmarks don’t matter as much as the overall experience of using a phone, but they do still matter enough that device makers sometimes put a thumb on the scale to improve scores. A recent report from Anandtech accused Huawei of configuring phones to produce artificially high benchmarks. Now, the maker of 3DMark has banned several Huawei devices in response.
The ban covers the Huawei P20 Pro, P20, Nova 3, and Honor Play. These devices will no longer appear in the list of top devices in UL Benchmarks apps (remember, UL recently acquired Futuremark). Essentially, Huawei was caught changing device performance when the phone detected a benchmark app. The phones would engage a much higher power limit for the SoC while running benchmarks in order to eke out a higher final score.
UL Benchmarks says it confirmed Anandtech’s testing on three of the four devices and took its word on the fourth (the P20). By using a private version of the 3DMark benchmark, UL was able to confirm that Huawei’s phones were operating differently when they detected the benchmark. In some cases, scores were 47% higher on the public app. This is considered cheating by UL because the resulting score is not indicative of the performance people can expect when using the phone.
UL says it reached out to Huawei, which has pledged to add a performance mode that lets users turn on the higher power limit. It’s unclear when that will happen or how it will affect benchmarks. We’ve pinged Huawei for more information and will update if we hear back.