As expected, the guys behind AnTuTu, didn’t take it well that companies are cheating on their benchmarks en masse, so they’ve decided to “fix” this problem with a new AnTuTu X Benchmark app (not to be confused with Basemark X), saying:
“A legal system operation optimization on devices is what we encourage, share the goal of tricking customers by cheating is what we are Firmly against”
The AnTuTu guys are angry at OEM’s for manipulating their benchmark and for good reason. If people start to think that all OEM’s cheat – and more importantly – are getting away with it, then before we know it, nobody will trust benchmarks anymore, therefore their benchmark apps will become irrelevant. So you can see why AnTuTu and other benchmark companies will not take this lightly. It threatens to put them out of business, because some of the benchmark companies have entire businesses around benchmarking.
AnTuTu doesn’t go into many details over how they plan to fix this, but they’re saying their new app will use some “camouflage tricks” that will help the app know when some OEM is trying to cheat with their device. Obviously, they can’t say exactly what it is, because then OEM’s might figure it out, and then bypass their tricks very easily.
Their app could for example check if the device itself is checking for the benchmark, which would be a tell that it wants to cheat on the benchmark. Then the app would send the device a message that it’s really some other ordinary app, perhaps some browser, so the processors reacts as it normally would, instead of getting a super-boost just for the benchmark itself.
I suspect OEM’s will try to figure out what it is and how to stop it anyway, and then we may start this all over again. But as long as AnTuTu, and others, make it as hard as possible for OEM’s to cheat, and then we find out quickly if they cheated again, then the problem may be reduced in the future.