SBC3 Wifi Testing

While our current wifi adapters do work well on the SBC, they are, unfortunately, rather slow. And though the speed, or lack thereof, of our current wifi adapter rarely has a notable impact on our projects, it does occasionally raise some issues. So after it started to negatively affect the performance and functionality of one of our recent projects, we sought to find something a little faster and, if possible, more reliable than what we had available.

But finding a fast and reliable wifi adapter isn't always as simple as going to the store and picking up the first thing you see. Most wifi adapters are created to work well with Windows and have very little concern for the select few users that might purchase their product to use with a Linux based operating system, such as the one that runs on the SBC. And though many of these same wifi adapters do work on Linux , the drivers to run them are rarely a shining example of perfection. The developers don't make enough money to justify perfecting the Linux drivers, they only want to get them working, not working well. The end result almost always has a handful of cut-corners and strange bugs popping up from time to time. So, inevitably, finding a fast and reliable wifi adapter means that you're going to have to do some testing yourself.

Wifi Testing Is Difficult

Wifi testing can be a very difficult thing to do. Wireless signals are not constrained inside a tiny wire, they fly around in the air and bump into walls, people, animals, and even each other. A simple analogy might be to think about sending data across an ethernet cable as if you put a letter on a train in Chicago destined to New York. Unless the train derails, there's a pretty good chance that letter you sent is actually going to get to New York. Whereas sending data wirelessly would be like strapping your letter to a stray dog in Chicago and telling it to go to New York. I don't have the same confidence that your letter is ever going to arrive in New York.

Different wireless protocols developed over the years have come to solve a lot of the issues that you can run into with wireless data transfers, but it's never going to be as simple as sending your data through a cable. Thes protocols need to implemented properly, and the hardware to transmit and receive signals can be very advanced. Small errors can render a device completely inoperable and utterly useless. And while large companies can afford to spend a million dollars, purchase some expensive equipment, hire some professionals, and build a warehouse to properly test every aspect of the devices in question, small companies and home users can't afford to be so lavish. Luckily, we don't have to spend a lot of money to narrow down the list of promising candidates and choose a good wifi adapter.

Wifi Testing Tools

When it comes to wifi testing, there are a number of different tools you can use. When it comes to wifi testing on Linux, that number suddenly shrinks... a lot... And when it comes to wifi testing on Linux using only console-based applications, well... you don't have a lot of options. Fortunately, we don't need a whole lot of options. The kind of things that I thought were most important to test were all things that could be tested with three very basic programs.

iperf

This program lets you test bandwidth between two locations over a set amount of time. You can also set other variables, such as packet size, TCP delay, or TCP Maximum segment size. For all intents and purposes, this forms the backbone of our testing suite. And though we're not going to bother fiddling with some of the more complicated options, we're going to be able to get a lot of important data out of the tests we run with this software.

To use iperf, we'll need to have a computer running Linux (preferably connected to our local network with an ethernet cable) in addition to our SBC. This Linux computer is going to be considered our "server". On the other end is our SBC connected to the wireless network via some wifi adapter you wish to test, which is going to be considered out "client".

From our server, open two terminal windows and use one of them to SSH into the SBC. On the server, run the command:

iperf -s

And on the SBC, run:

iperf -c [ip address of server] -t 1200

(If you don't know the ip address of the server you can find it by opening a third terminal window and typing: ifconfig )

This will run a 20 minute long test. (you can change the length of the test by increasing or increasing the number 1200 above.) At the end, it will simply tell you the bandwidth over the network. For us, however, this program fills a vital role in flooding the network with wireless signals. If the wifi adapter is going to fail, it's going to fail during a time like this, when it's maxing out it's capabilities, and we're going to be able to see it.

iptraf

This is one of the programs we're going to use to monitor the connection we've set up with iperf. It has a pretty, colourful interface, at least when you compare it with the other programs we're using, and lot's of nice options to monitor different parts of the connection. Personally, I found the "Detailed Interface Statistics" to be the most useful, but you may want to explore the program a bit and learn some of its features.

You can get a lot of data out of this program, but to be totally honest, I just used it as a way of checking to make sure that the wireless adapter hadn't failed catastrophically. That is, if the numbers stopped changing for a period of time, I assumed that the connection had been completely lost and I took note of it. And while I admit that this is a very primitive way of monitoring the network, it did in fact give me the results that I needed.

watch 'netstat -s | grep retransmited'

This is actually more of a hack than a real program, but it works very well for what it is. By running this command in the terminal, you can see how many packets have been retransmitted since the device has been powered on. This number will update once every two seconds, so you can simply start it up and leave it running. Pay close attention to how many packets have been retransmitted. If the number climbs unexpectedly, you may have a problem with your wifi adapter. Typically, I found that good wifi adapters had 100 or less retransmits over a 20 minute test, while bad wifi adapters had well over 1000!

Collecting Data From Your Tests

You can experiment by running these tests at different distances and taking note of how the performance is affected. For each wifi adapter I tested, I ran three tests at short range and then three more at the longest range I could reach without having the wireless adapter completely disconnect. During the tests I took note of three things: the average bandwidth, the number of retransmits, and the number of times that the wifi adapter failed catastrophically. (By this I mean that the wifi adapter lost connection with the server.) Depending on how much testing you want to do, you could keep track of other things too, such as minimum bandwidth and maximum bandwidth. However, it's difficult to try and keep track of too many things at once without completely automating the tests, so I tried to gather only the most important data. Though, If you're planning on doing a lot of tests, you may want to find some kind of way to automate the tests anyway.

Testing Conclusions

In the end, these were some of the results that I gathered from some of the wifi adapters that we purchased (Each number is taken as a average of all tests):

Adapter Name	Retransmits	Connection Speed (Mbps)	Catastrophic Failures
EDUP EP-N1528	259	10.2	0.67
EDUP EP-MS150NW	13871	3.62	4
Phidgets	98.7	1.77	0
Dlink DWA-140	3454	5.9	1.33
Patriot Wireless N 300Mbps	7.33	18.5	0
ASUS USB-N13	1064	10.0	1.33
No-name Wireless N 150M Adapter	5182	4.99	5
3703_0 - Wi-Fi USB Adapter 802.11n	19.7	18.7	0

Right away you can see that a few of the devices stand out as being better than others in terms of reliability or speed. For example, the current Phidgets wifi adapter, the Patriot adapter, and the 3703_0 all had relatively low retransmits and zero catastrophic failures. Though, it should be noted that the Patriot wifi adapter and 3703_0 were considerably more reliable, only suffering an average of less than 20 retransmits per 20 minute session. The EDUP EP-MS150NW and No-name 150M adapters were notably worse, both having thousands of retransmits and multiple catastrophic failures per 20 minute session.

In addition, we can see that the Patriot adapter and the 3703_0 were easily the fastest adapters we tested, with the Edup EP-N1528 and the ASUS USB-N13 following at half the speed. The Phidgets adapter performed very poorly, achieving one-tenth the average speed of the Patriot adapter and 3703_0.

What may become immediately apparent to someone looking at these results is how vastly different they all seem to be. However, it seems that the performance of a wireless adapter is closely related to the Linux driver associated with them. By using the command "lsusb" in the terminal while we have one of our wifi adapters plugged in, we can see what driver they are using.

Adapter Name	Driver
EDUP EP-N1528	RTL8192CU
EDUP EP-MS150NW	RT5370
Phidgets	ZD1211B
Dlink DWA-140	RT3072
Patriot Wireless N 300Mbps	RTL8191SU
ASUS USB-N13	RTL8192CU
No-name Wireless N 150M Adapter	RT5370
3703_0 - Wi-Fi USB Adapter 802.11n	RTL8191SU

With this new information we can see that the 3703_0 uses the same driver as the Patriot adapter, which might explain why they both have such good performance. Additionally, the No-name 150M adapter and the EDUP EP-MS150NW adapter both share the same driver, which might explain why the both have such poor performance. Finally, the ASUS adapter and the EDUP EP-N1528 adapter also share the same driver.

From the information above, it can be concluded that the driver has a large impact on the performance of a wireless adapter, but it does not necessarily decide it's fate. For example, the EDUP-N1528 adapter had a much lower retransmission rate than the ASUS adapter did. So although it might be worth testing several different adapters that use different drivers, it may also be worth testing devices that use the same adapter.

Ultimately, if performance and reliability is important to you, it's worth testing a few different products. Because, although the tests we ran on these devices were not necessarily thorough, they helped us narrow down the list of suitable devices, letting us cut out extremely unreliable adapters such as the No-name Wireless N 150M, Dlink DWA-140, ASUS USB-N13 and EDUP-MS150NW.