Froyo “450% Faster”? What's it really mean?
Anyone who’s remotely into Android knows that Froyo (2.2) is coming out soon. The rumors (along with the most logical date) indicate Wednesday, May 19. Why that particular day? It’s Google IO 2010, and it would just make sense. So that’s great, if you’re running a Nexus One. Everyone else will probably get it in the next… 6 months or so, should your vendor decide to let you upgrade. That being said, I ran across an article this past evening that caught my eye: “EXCLUSIVE: AndroidPolice.com’s Nexus One Is Running Android 2.2 Froyo. How Fast Is It Compared To 2.1? Oh, Only About 450% Faster”
Supposedly, they got a Nexus One, running a specific test, to go from 6.5 MFLOPS to 37.5 MFLOPS. So yea, the numbers say that is a huge increase, but is it really? First, They don’t say if they used the same exact hardware for both tests, and as this is still pre-release on Froyo, I seriously wonder if they aren’t two different units. Though the units could be the same, the might not be.
Second item of concern is the fact that this is one single speed test. It is actually fairly easy for a JIT layer to be optimized in one specific direction. To get a more accurate view of the system speed, you should be running a dozen different types of test. Preferably you’d be running tests that emulate user usage. For example, parsing out a large amount of JSON data takes quite a while and is something many applications do.
Third item of concern, which is in close proximity to number two: will the end user actually notice any speed difference. Ok, lets be fair, if the system is super uber faster in 2.2, the user might not notice it, but application developers could take advantage of that. Being able to do more with less is always a good thing. In reality though, if you’re application takes a second to warm up, you most likely won’t notice it being all that much faster. Of course there are numerous other factors that are involved with applications taking their sweet time. Loading data from memory (card) isn’t going to go any faster than the card supports simply because the JIT is faster. If the application uses a network connection, that isn’t going to go any faster either.
In the end, I’m hopeful, but skeptical. I don’t want to say that these tests are crap, but without a lot more information… there isn’t much to go on other than their word. I’m also extremely skeptical of any tests claiming a system is massively faster. Simple tests like this are super easy to fabricate one way or another. I’m not saying they didn’t actually do a test, but the test could have been a setup. With enough time and the right “test”, I could prove to you that a walrus is faster than a Boeing 747 jumbo jet. Completely absurd, yes… but the point is that marketing groups have been doing this sort of “test” for years. Anyone remember the HP bulletproof server? Yea… nuff said.