OnePlus has been accused of fiddling benchmarking software with the new OnePlus 5 smartphone, according to a report by XDA developers.
Update (June 23, 2017): Since the publication of this article, OnePlus has responded with several comments regarding the allegations of benchmark fixing.
The first was sent to XDA Developers, and reads:
“
OnePlus then sent the following statement to both TrustedReviews and XDA Developers:
“We have set the OnePlus 5 to run benchmarks at a high-performance level that is both natural and sustainable for all devices, media and consumer, so that users can see the true potential of the device, when running resource intensive apps and games. At no point do we overclock the CPU, nor do we set a CPU frequency floor. We are confident our approach best displays the true performance capabilities of the OnePlus 5.”
XDA Developers has since updated its original blog post with its own response to the OnePlus allegations, confirming that OnePlus is correct in stating that there is no overclocking or CPU frequency floor. The article also notes that the “product itself is good, performance is phenomenal”. However, the blog maintains that the way the OnePlus 5 is set up to handle benchmarks is misleading:
“The fact remains that benchmark producers, Android engineers and enthusiasts see these mechanisms as unfair, unrepresentative of the user experience, a bit deceptive and, ultimately, a waste of resources that could have gone to another aspect of product development. If you want to see just how hard the OnePlus 5 can push its hardware, then you’ll likely agree with OnePlus’ decision.”
It continues: “If you want to see how the OnePlus 5’s combination of hardware and standard software compare against other devices with similar hardware, you are out of luck, because manipulating CPU frequencies in such a way makes scores wholly incommensurable.”
Original article, as published on June 21, 2017:
One of the ways to test a phone’s computing heft is by using benchmarking software. These are apps that get a smartphone to perform difficult tasks in order to get a rough idea of the device’s maximum processing performance. Serious Android fans often consider benchmark test results when buying a new phone, so it’s important that the scores aren’t misleading in any way.
But Android developer hub XDA Developers has published a lengthy blog post that accuses OnePlus of “inappropriately manipulating benchmark scores” with the OnePlus 5 – a phone that only launched yesterday. The review handset –which XDA received ten days before the phone’s official debut – features a so-called “cheating mechanism” that is described as being “blatant and aimed at maximising performance”.
“Unfortunately, it is almost certain that every single review of the OnePlus 5 that contains a benchmark is using misleading results,” the report reads. “This is an inexcusable move, because it is ultimately an attempt to mislead not just customers, but taint the work of reviewers and journalists with misleading data that most are not able to vet or verify.”
The blog post describes how a OnePlus 5 with the cheating code enabled keeps the clock rate artificially high during benchmark tests, as the following charts depict:
The problem with this is that keeping cores at their maximum clock rates isn’t sustainable, because it can significantly affect core metrics like battery life and device temperature. With normal phone usage, a core would hit the maximum frequency intermittently, allowing the cores to have downtime to keep the device cool and running efficiently. The post explains: “These altered results are not representative of the real world performance of the OnePlus 5 in any way, as they are reflecting a peak and otherwise-unattainable performance of the device under artificial conditions and without constraints.”
According to XDA Developers, the following benchmark applications are affected:
- AnTuTu
- Androbench
- Geekbench 4
- GFXBench
- Quadrant
- Nenamark 2
- Vellamo
This isn’t the first time OnePlus has been accused of this either; XDA Developers called OnePlus out for employing a similar tactic back in January this year.
“It is a bit upsetting that it has gotten to the point where we have to call out the same company twice for manipulating benchmark scores,” the post reads. “It is worse this time around, with the apparent purpose of inflating scores produced by reviewer handsets.”
It’s worth noting however, that the software running on the OnePlus 5 was pre-production, and it’s entirely possible that customers won’t get such a device – not that this makes the alleged deception any easier to swallow. The article reads: “We are confident the culprit code will be removed from consumer builds following this report and our conversations with OnePlus representatives. While we do not believe this feature article necessarily should alter your perception of the hardware itself, it is right for it to nudge your opinion of the company given it is their second transgression.”
We’ve asked OnePlus for comment on the matter, and will update this article with any response. The handset goes on sale officially on June 27, 2017.
Related: Best Android phones
Source: XDA Developers
What do you think of these allegations? Let us know in the comments.
Editorial independence
Editorial independence means being able to give an unbiased verdict about a product or company, with the avoidance of conflicts of interest. To ensure this is possible, every member of the editorial staff follows a clear code of conduct.
Professional conduct
We also expect our journalists to follow clear ethical standards in their work. Our staff members must strive for honesty and accuracy in everything they do. We follow the IPSO Editors’ code of practice to underpin these standards.
Editorial independence
Editorial independence means being able to give an unbiased verdict about a product or company, with the avoidance of conflicts of interest. To ensure this is possible, every member of the editorial staff follows a clear code of conduct.
Professional conduct
We also expect our journalists to follow clear ethical standards in their work. Our staff members must strive for honesty and accuracy in everything they do. We follow the IPSO Editors’ code of practice to underpin these standards.
