Redmagic's benchmark shenanigans: are smartphone makers secretly cheating?

Smartphone performance benchmarks are under intense scrutiny after UL Solutions delisted results from RedMagic phones, raising serious questions about manufacturers gaming the system.

The ul fallout: redmagic's scores scrubbed

The story began with NotebookCheck reporting the removal of RedMagic 11 Pro and 11 Pro Plus benchmarks from UL Solutions’ database. This isn’t a minor hiccup; UL’s tools—3DMark, PCMark, Testdriver—are staples for reviewers and consumers alike. It’s a significant blow to the credibility of performance claims across the industry.

The ‘cheating’ revelation

Japanese YouTube channel Saityo dug deep, uncovering a startling discrepancy. RedMagic was reportedly boosting its scores by optimizing the phone’s hardware specifically for the 3DMark benchmark, a technique known as ‘benchmark optimization’. They were essentially exploiting a loophole, triggering a custom performance profile when the test software ran.

The rules, broken

UL’s benchmark rules are clear: no platform-specific optimizations are allowed. The platform shouldn’t even detect the benchmark launch. Yet, RedMagic was actively modifying the phone’s behavior to maximize its score, a blatant violation. The core of the issue lies in this automated adjustment– it’s a fundamental misinterpretation of what a benchmark should measure.

Redmagic’s defense: performance modes

RedMagic countered, arguing that its ‘Diablo Mode’ and other performance settings are designed to dynamically allocate resources based on application load. They claim this isn’t cheating; it’s simply providing a more responsive gaming experience. However, critics point out that this optimization is triggered automatically, circumventing the core principle of a standardized benchmark.

A gray area?

The debate isn’t black and white. While optimizing for high scores is undeniably questionable, RedMagic’s approach – automatically tailoring hardware for a specific test – exposes a critical flaw in how benchmarks are interpreted. It highlights the tension between delivering a genuinely smooth experience and providing an objective performance measurement. The fact that RedMagic’s optimized performance felt great in real-world usage only adds to the complexity.

The verdict?

Let’s be blunt: this is a case of deliberate deception. UL's rules are there for a reason – to provide a consistent and reliable measure of performance. RedMagic’s actions undermine the entire process. While manufacturers routinely tweak their devices for optimal performance, manipulating a benchmark to achieve an artificially inflated score is simply unacceptable. It’s a race to the bottom, and consumers deserve better.