Introduction
You see Wireless N and Bluetooth 3.0 slowly creeping on the cellphone specs pages and it was about time we do a dedicated test of how good they are in reality. Every geek loves to get the latest and greatest in mobile tech but it’s even sweeter when you have the hard evidence to back up the upgrades brought by the new version numbers.
We rolled up our sleeves and prepared for testing the Wireless N implementation in some of the latest and greatest smartphones on the market. We wanted to find out just how faster or wide reaching it is compared to the good old G standard that’s been around for what looks like ages.
Bluetooth 3.0 is also growing in popularity and for a technology that’s been running at snail speed ever since it was introduced, we expected even greater improvements. We brewed a lot of fresh coffee and prepared for a long shift full of rolling tests and crunching numbers.
We picked four of the most popular phones that support Wireless N – the iPhone 4, the Nokia N8, the Samsung I9000 Galaxy S and the Samsung S8500 Wave. The variety will ensure we get results that are OS independent. We had Bluetooth 3.0 enabled phones lined up too, but as it turned out, we had jumped the gun on that one.
Apple iPhone 4 • Nokia N8 • Samsung I9000 Galaxy S • Samsung S8500 Wave
On paper Wi-Fi 802.11n promises a huge increase in bandwidth (up to 300Mbps vs. 54Mbps for the b/g standards) and improved performance at longer ranges. Bluetooth 3.0 similarly promises faster speeds (24Mbps vs. 3Mbps for v2.1+EDR) and adds better power management along with low-latency connections.
However we were in for a surprise. Right from the very first sample tests we did on both the Wireless N tech and Bluetooth 3.0 we knew something was wrong – we weren’t getting the advertised improvements. Now how about that?
Both of these latest wireless technologies over-promised and under-delivered, at least as far as mobile phones are concerned. We obviously needed to include more test scenarios and test the tech on more phones – just to be sure we’re not missing something.
And there we went – we curbed our enthusiasm and started slowly, approaching the tests more carefully. And as we dug deeper, things got ever more confusing.
We bet you’re dying to learn the hard figures behind these proclaimed new technologies. Well, be our guests and jump on to the next page where we start off with the Wireless N Olympics.
Wireless N test setup
For the Wi-Fi tests we did our best to eliminate all variables save for Wi-Fi itself. We ran our own web server in the local LAN and we used a Netgear WNDR3700 router for spreading the Wi-Fi love around.
Our first test included downloading a 30MB file with the phone’s web browser. We picked two testing positions – near the router (about 1.5m away from it) and in another room with walls between the phone and the router (Wireless N promises better performance than b/g at the same distance given the coverage is good enough).
Since Wi-Fi performance can be affected by neighboring networks, we ran multiple tests and took the best one. Tests were conducted with a Samsung I9000 Galaxy S, a Nokia N8, an iPhone 4 and a Samsung S8500 Wave – different platforms, different OSes.
We also did a second test by hosting some static web pages on our LAN web server to see if the new standard gave any speed improvement in terms of latency. The web page loading also forced the phone to load multiple small files instead of just one big file (which also might affect the performance). This test was conducted at 5m away from the wireless router.
The three static pages are actually snapshots from our site, so they are real-world stuff. We recorded the times the browser needed to open the three with an accuracy of up to 1/30 of a second (we shot the whole testing on 30fps video, if you’re curious).
Wireless N test 1 – measuring the throughput
So here are the aggregated results from the first test – the file download. We only list how much better or worse is using Wireless N over the standard-issue G protocol (0% means no change in throughput in the compared modes). We used two Wireless N modes – the "neighbour-friendly" Wireless N 130Mbps mode and the "maximum performance" Wireless N 300Mbps mode.
Bear in mind, these numbers won't allow you to compare the wireless throughput of the different phones. They only show the way the throughput of any individual phone changes when you go through the available WLAN modes.
Throughput results – the table shows the bitrate change in percent (0% means no change)
Let’s start with performance near the router – Wireless N at 130Mbps was within 15% of WirelessG, sometimes posting a worse result (the Nokia N8) and sometimes both tests were exactly the same (the iPhone 4).
Wireless N at 300Mbps showed a modest improvement over WirelessG (about 10%) and then there was the Nokia N8, which also posted a slightly slower result. The iPhone 4 on the other hand showed a huge improvement in this best case scenario (though on average it hovered around WirelessG speeds).
At close range the difference between WirelessG and the two Wireless N modes is pretty much negligible. So, it’s down to the second test to decide the winner (the second test is also more realistic in terms of usage anyway – people rarely stand this close to the router).
Although Wireless N doesn’t increase the range of Wi-Fi, it should offer a better throughput than the G protocol at the same distance.
Our tests show that with mobile phones this is not guaranteed and strongly depends on the Wireless N mode used on the access point.
At our furthest test location the phones performed consistently slower at Wireless N 130Mbps than on G protocol except the iPhone, which scored the exact same result in both cases.
But switching to the 300Mbps Wireless N mode gave us the improvement we were looking for. The iPhone 4 again didn’t see a difference, while the Nokia N8 managed a 12.5% improvement (but still remained under its G mode score).
With the Samsung Wave and Galaxy S however, the 300Mbps mode really mattered. The Wave saw a 60% improvement, while the Galaxy S managed double the bitrate as compared to the 130Mbps mode.
The cream of the crop
We guess you're dying to learn which phone has got the fastest wireless throughput. We won't be publishing any detailed numbers because that will skew the purpose of these tests, but we'll just mentions some.
Closer to the wireless router, the phones with the greatest throughput were the Samsung Galaxy S (20Mbps) and the Apple iPhone 4 (17Mbps). The Galaxy S was the best performer in G-only mode as well (18Mbps).
However further away from the router, it was the Nokia N8 that came on top. Although it didn't show much benefit from the Wireless N mode, the N8 had the most consistent throughput (between 10 and 14 Mbps) that kept its pace from all locations and in all test modes.
Wireless N test 2 – loading a series of web pages
And here we come to the second test, which is actually closer to real-life usage - the web page load times.
We only list how much better or worse is using Wireless N over the standard-issue G protocol (0% percent means no change). We used two Wireless N modes – the "neighbour-friendly" Wireless N 130Mbps mode and the "maximum performance" Wireless N 300Mbps mode.
Unlike the previous tests we did the test from only one location – about 5m from the wireless router in direct line of sight.
Bear in mind, these numbers won't allow you to compare the wireless throughput of the different phones. They only show the way the throughput of any individual phone changes when you go through the available WLAN modes.
Page load times – the table shows the time difference in percent (0% percent means no change)
When it came to loading a web page we got mixed results – three of the phones (Nokia N8, iPhone 4 and Samsung Wave) showed a slight improvement (5-13%) while the Samsung Galaxy S took a big hit.
Using the 130Mbps Wireless N mode, the droid browser slowed down by 8%, while the 300Mbps mode made things even worse, a big 30% hit.
Obviously when downloading big files, Wireless N makes a clear difference (if the router is set up correctly that is). But when it comes to loading pages, the differences of around 10% will probably be masked by the uncertain networking conditions of real-world Internet usage.
As for 130Mbps vs. 300Mbps, the supposedly faster mode either made no difference (5% is below the threshold of what users will notice) or made things slower.
Keep in mind that we’re not comparing the different browsers – if you want a browser shootout, check out our touch browser mega shootout. We’re comparing Wi-Fi performance here – how much the same browser will speed up (or slow down) when we change the Wi-Fi mode.
Tips to get the most out of your WirelessN router
During the tests, none of the phones went over 20Mbps transfer speed anyway, but that’s to be expected. The lesson here (one we learned the hard way as we had to redo the tests) is that how you set up the Wireless N router matters a lot!
The 300Mbps mode worked better for us, especially if the router is in the other room. There are other things you need to consider though (note that some routers might perform differently).
First off, you need to use WPA2/AES encryption or your Wireless N network will work at 54Mbps. Second, Quality of Service (QoS) has to be enabled to hit the higher speeds (our test phones performed worse with QoS off).
Last but not least, if a Wi-Fi b/g device connects to a Wireless N network simultaneously with your N client, the network automatically throttles down the down to 54Mbps to accommodate the new device (at least while the b/g device is actively using the link).
This is important to keep in mind if you have multiple devices of various generations connected to the same network. To solve that you can add a simple separate 802.11g access point (not a router) for those G clients.
Bluetooth 3.0 labels and why they matter
We researched the Bluetooth 3.0 issue and the reason for the low speeds we were getting became instantly clear – it’s a simple case of confusing labels. You’d think that Bluetooth 3.0 is Bluetooth 3.0, right? Wrong.
The Bluetooth 3.0 specifications detail three things – Unicast connection-less data, Enhanced Power Control and Alternate MAC/PHY.
The Unicast part specifies a way to send a little data without much delay (e.g. a remote control) and the Enhanced Power Control keeps a tighter reign on the transmitting power and makes the device a little more power-efficient.
But it’s the Alternate MAC/PHY section of the specifications that’s interesting – and also the section that’s not mandatory to get a Bluetooth 3.0 certification. It allows two BT3.0-certified devices to do a handshake over a Bluetooth link and then switch to a 802.11 link (the base technology for Wi-Fi) to achieve speeds of up to 24Mbps.
Devices that support Alternate MAC/PHY are certified as “Bluetooth 3.0+HS” (High Speed) and ones that don’t are certified as just “Bluetooth 3.0” and are limited to regular Bluetooth transfer speeds (no Wi-Fi magic).
A bit more digging reveals that Android 2.2 (and below) doesn’t even support Bluetooth 3.0 – the BlueZ library that it uses is old and works only up to Bluetooth 2.1+EDR. You can check Bluetooth support info for the other Android versions over at their dev site (there’s no info on Gingerbread though).
It’s possible that manufacturers (like Samsung) use a different software stack, but the transfer between the two Bluetooth 3.0 certified Samsung droids we tested was going at v2.1 speeds, so that wasn’t it.
The Bluetooth SIG website settled any remaining doubts we had. None of the phones we checked were Bluetooth 3.0+HS certified, they support just the vanilla v3.0. This put an end to this speed test – 24Mbps just wasn’t going to happen so when you come to pick your next uber-smartphone you can safely ignore the Bluetooth 3.0 specs as long as they don’t list HS explicitly.
Final words
Consumers like bigger numbers – it’s a fact of life. If you sit around the Internet fireplace long enough you’ll hear all sorts of stories of companies picking higher numbers for marketing reasons more than for actual benefit of the end user.
One tale of the Internet lore holds that Microsoft called their second Xbox “Xbox 360” so that it’s not Xbox 2 vs. Playstation 3 (which would seemingly put Sony’s console one generation ahead). Canon and Nikon have a similar game going on with their competing products too.
We don’t know if that’s true or not but as far as Bluetooth 3.0 is concerned, that’s pretty much the case – jumping from version 2 to version 3 is more for show than because of some tangible improvement.
New software might enable Bluetooth 3.0+HS on some phones (if the underlying hardware supports it) but right now we’ll have to settle for Bluetooth 2.1+EDR speeds no matter what the box label says.
As far as the latest Wireless N connectivity is concerned, we did see some improvement but the results weren’t as definitive as we thought they would be judging by the specs on paper. It was better in some cases and worse in others. In any case, you won’t get six-time speed up as the 54Mbps vs. 300Mbps numbers might lead you to believe. It’s nowhere close to even two times in fact.
Overall, Wireless N mode will help out in areas where the G protocol bitrate drops substantially, which can be quite useful if you’re using only one router to cover your home.
It pays to use a Wireless N router and phone – your laptop will certainly enjoy a speed-up from the new standard but if your phone has only Wi-Fi b/g support, it will drop the speed of the entire wireless network for N clients as well.
While both Bluetooth 3.0 and Wi-Fi 802.11n have their place in the mobile world, you won’t be missing out much if your phone doesn’t support either of them right now.
No comments:
Post a Comment