Introduction
Back in October I wrote a post about testing your mobile application on different networks and I promised a few follow-up articles. Unfortunately, I have been sidetrack with a lot of other work, and finally I have some time to get back to addressing this issue. This article will go over some results of data mining on different networks, and where the differences are. A future post will cover some ideas of how to simulate different networks for testing.
I spent my ‘free’ time for the past few months researching OFDM, specifically how different carriers have implemented it, and one thing became painfully clear. Outside of the basic theories of multiplexing, everything thrown into the implementation is proprietary. I spent most of my time trying to determine differences between Sprint and Verizon’s network, and even what they put in their packet headers is not publicly available. This isn’t to say the information is not out there, but it makes analysis much more difficult. This also doesn’t mean I wasn’t able to see how the information was transmitted, or even what was transmitted.
Data Mining
For this particular exercise, I decided I needed to limit what I was looking at. Since I couldn’t officially determine how the data was being encoded and transmitted, I decided to look at one device, over two different networks, and see what if any differences I could find. In this way I was hoping to limit my variables, and focus on what I was interest in, the data. I decided to go with Android device, as these are very ‘open and friendly’ devices, which run over multiple networks.
I happen to be lucky enough to have several nerdy friends with rooted phones. After asking around some about what phones everyone had, imagine my happiness when I learned that two of them had a Samsung S4; one with a Sprint plan and one with Verizon.
Shark for Root was installed, as we decided we would monitor the network traffic, and then see what different data was transmitted. After some discussion, we decided that we would run 3 different tests.
- Run Google Maps over 4G
- Run Google Maps over 3G
- Run Google Maps over WiFi
We set all of the phones to the same Android version, and ensured that Google Maps was at the latest available. One of the first things we learned was that Shark for Root does not capture traffic properly over 4G. So with that first test a bust we relocated to where 4G service wasn’t available.
After several queries and searches, we had more than enough data, and we switched over to wifi. We re-ran those same searches and queries again. Remember, we did both of these for an Android phone on Sprint and Verizon plans.
After working in the mobile space, one thing I’ve unfortunately learned is that data is not required to be encrypted over the wire by any mobile OS. Not only that, but most applications don’t bother encrypting their data; if it’s not a requirement, why waste the time/effort. I was both pleasantly surprised and annoyed to see that Google Maps did in deed seem to be protecting the data sent back and forth. They used secure API calls over SSL, and so most of the data we looked at wasn’t in any sort of clear text.
Data Analysis
What we instantly noticed though was that the data being sent from the Verizon phone was about 1.5 times larger than the amount of data sent from the Sprint phone. This was over both 3G and WiFi. Now, we figured they could be multiple reasons for this.
- Each phone, while rooted, still ran the original OS on the phone, which had it’s own tweaks done be each carrier. This means it’s possible that the OS itself was responsible for sending additional or leaving out data.
- Instead of the OS, it could be that the network itself required additional data
- Different compression could be used for the same data, account for the data size difference
Do you have any other ideas as what could be responsible for this data difference? We were stumped pretty well.
In addition to the difference in data sizes, we also noticed that different header fields were present from WiFi from 3G. Without going into to many technical details, this made sense to us. Different protocols are in use whether we are looking at strictly web or telephony signals, from UMTS for phone to TCP/IP for web. We also noticed some differences between the Verizon and Sprint information sent, and more so than just destination information. This seems to indicate different protocols being used by each carrier to process the data from PDCP to PPP. While this shouldn’t effect the data in any way, it does become more obvious what differences a network change can make, in addition to the way data is transmitted.
Conclusions Drawn
There was an incredible amount of data obtained from these tests, and while we couldn’t draw any absolute conclusions about how Verizon or Sprint handled data differently, it was very apparent that there are some differences. The big question still remains…what does this mean for testing? Is it worth testing over one network vs another? Is it enough to just test over WiFi? Come listen to my talk at STAREAST on May 7th, and hear some more details about setting up Shark for Root, and how the data was analyzed. I will also be posting a followup for some next steps following the questions thrown out here. Have some comments, questions, or thoughts? Please leave them below.
