Mobile Viewpoint Reviewed: A (Literal) Road Test

While we were aware CPT could iron out any link loss on our own test, we opted for the 2-second latency setting so that our test actually measured the carrier capabilities a little more closely.

With that description I should now turn to the results.

You can see the map of our testing course in Figure 4. In advance we knew, by virtue of being local, where we expected to see dropout, and indeed our best guesses proved to be accurate. Our guesses collectively anticipated a drop out at point 12/14 as seen on the map, and at a point halfway between 20 and 22. The trial also showed a dropout at 2.

Mobile Viewpoint Testing Route

Figure 4. Map of the testing route

Watching the video of the route is an absolutely clear way to see how successful the link was in any practical sense. Bear in mind that this was the original video from the receiver re-encoded to FLV and then uploaded to my YouTube account, where it was almost certainly re-encoded, and definitely re-muxed. While I think it looks good, it is important to be clear that this is a generation or two away from the source recorded on the receiver’s DVR. (If you look closely in the first few moments you may see a crazy dog running down the centre of the highway too!)

So from a subjective point of view we can look at the video and really get an idea about how effective this technology can be.

A Scientific Measure of Cellular Link Quality

But I wanted to go a little further than that. With some amazing technical journalism in the pages of Streaming Media magazine from the likes of Jan Ozer and Tim Siglin, I wanted to actually come up with a scientific, quantitative way to measure the link quality.

Since the bitrate of the video stream as it was recorded at the receiver drops when the cellular link faded, I decided to try a wire-level packet analysis. In my mind this analysis is a rigourous as any other method I could think of and I will attempt to use this method for all and any cellmux tests in future.

Essentially, once the recording was saved on the receiver’s DVR, I could connect to the DVR feature using VLC or QuickTime and an RTSP:// url. As the stream then played back to my machine, I used WireShark to capture all the packets specifically between my laptop and the receiver’s DVR source. With just that conversation captured—in essence every video packet played back—I could then use Wireshark’s IO Stats grapher to plot the data payloads by time. This is about as accurate a measure of the bandwidth of video that you could get, literally measuring the size of each datagram and plotting them against time. It doesn’t allow for variance in the Wi-Fi signal between my laptop and the receiver, but the range was about 4m and the normal signal quality of this link is far higher than 2.5Mbps so while I should setup a control next time, I seriously doubt my Wi-Fi QoS affected the results I captured.

Mobile Viewpoint testing results

Figure 5. Testing results. See paragraphs below for key.

You can see the results in Figure 5. The top two graphs are taken from LogYourRun. The first shows velocity with higher points on the Y axis representing slower velocity. So from a standing start we left the city and averaged about 60 mph. The small slow down at 15 minutes is the Lewes roundabout.

The middle graph is the altitude in meters. The roundabout (point 16 on the map) represents the lowest point on the journey. We found this useful both for aligning the data on the charts, but also to see how the performance was effected by the altitude—particularly in the last 10 minutes when, despite being high, we still saw significant drop in signal.

So the final graph shows the video quality as measured by Wireshark, and the line—pretty accurately in my opinion—shows the variance from the target 2.5Mbps. Note that at many points the signal would have dropped and the adaptive bitrate encoding model in use over the link would have downgraded the encoding profile to fit within the available link capacity, so long as there was actually a signal available. This would have increased artifacting/pixelation and so on, but the fundamental flow of video would have remained solid throughout.

I also collected a number of graphs from a service called Sensorly that crowdsources (via mobile apps) mobile network operator coverage graphs, and it was good to see that the coverage an operator’s footprints that Sensorly anticipated were in line with what we saw on the UI of the device.

Around point 2 we were in the Brighton suburbs, and accordingly some of the quality of signal loss would have likely been down to interference from buildings and contention with other service users. Between there and point 12 the signal was good; as we passed the University with its population of 7000 to 8000 the cellular quality was good and very stable (I perhaps the small regular bumps every 30 seconds or so are down to the handover from mast to mast). At point 12 dense trees, unfavourable geographics, and an extremely low population density lead to a loss, followed by a dip at the roundabout at point 16. Interestingly, as you can see in the video, there was a large police incident support unit near the roundabout, and I noticed it was covered with cameras. I wondered if the dip at the roundabout could have been contending with some comms on that police unit.

As we return to Brighton we again pass the point 12 with signal loss; Tim and I got very excited at the time about how the signal loss at this point is loosely symmetrical around the roundabout point.

The one signal drop that confused us was the one at point 21. As we crested the hill over to Woodingdean we suddenly had line of sight to a very close transmitter, however it seemed to be that the Mobile Viewpoint lost the signal completely as we came very close and passed the tower. We discussed this in some depth. Michael, Tim and I all felt that this was a problem with the handover between the cells. We think that as we left the A27 road and headed up the hill we gradually got further away from the tower we were connected to, which would have caused all our modems to increase their power, only to suddenly crest the hill, and be presented with a much more powerful mast. The theory is that that mast would have then immediately demanded that our modems dropped their power significantly so as not to drown out other signal in the area. This quenching could have caused problems for our QoS-sensitive data application and caused the underlying link failure we saw.

So there were no real surprises that the link had a few drops at all, but none of those drops were even a minute long, let alone two minutes, so should we have used the CPT setting on the system we should have achieved a continuous 2.5Mbps video contribution feed—more than enough to have covered the Olympic Torch continuously.

The Verdict on Mobile Viewpoint

While this is an isolated experiment at the moment, now we have a model I look forward to benchmarking other rigs in this way.

There are several other bits of practical information that have been fed back by Tim and Michael that should be included in this article:

Tim particularly liked the antenna. Despite its size he had, until he saw the mast, planned to shield his back with aluminium foil. While he is (often) a little left of field, Michel assures me that Tim is not alone in being concerned about cellmuxes’ radiation and its proximity to the operator.
The kit is rugged. It is far from the smallest of these devices, but there is no doubt that its build is operationally focussed. It is rugged, easy to control, and has some nice features on the UI such as ‘press and hold’ on all buttons and touch screen to avoid accidental knocks.
It can run on mains, using a Vpower camera battery adaptor, or (as we did) off an inverter in a car cigarette lighter socket.
There were one or two minor issues: A few of the cable connectors seemed to have been added as an afterthought, or the layout of physical connector interfaces is simply considered for a specific use case—battery, SDI, in backpack—and so while the SDI and power sockets were on the front, we found the composite connector (which needed a BNC barrel adaptor) on the side of the unit, and this meant that by the time we connected our camera the cable was pressed tightly against the backpack. The Ethernet cable socket was on the other side, and this again would have made it awkward to connect anything via Ethernet while using the case.
The antenna makes the pack top heavy when extended, so careful adjustment of the straps is needed to ensure the antenna doesn’t pull the whole unit back too hard, like a small child hanging off your rucksack.
With the telescopic antenna 2m above your head, a) you look like an uber-geek, b) it would be a bad setup for filming lightning strikes while standing on a hill, and c) you need to remember it’s up there before you go under any low wires or entrances.

The final use comment came after we tried to run a DVD through the system to test a higher quality source than my Handycam. It was about 9 p.m. by the time we set this up. We fired up the unit indoors on the table in my room and started the stream running.

Surprisingly I could really feel the effect of the radio emitter. Frankly it felt like my fillings were trying to turn around and immediately brought on a mild headache, something like the effect of high-tension overhead power lines. Within a few minutes of the test both Tim and I were feeling really uncomfortable, slightly edgy and decided to turn the system off. The overall experience was quite unsettling. During the day there were a lot of stimuli to distract us, but in this evening test it was quite clear to me that having half a dozen radio transmitters blasting at full signal right near my head unarguably has an effect on how I felt.

Tim made a very interesting observation: If you were using one of these for an event like a press conference, most of the people there are going to feel uncomfortable simply because of the radio transmission – and this is a very real negative side effect of these type of devices. Admittedly this is extremely hard to objectively measure, but once you have experienced it you won’t forget it.

Thankfully for Mobile Viewpoint the telescopic antenna really mitigates that issue, but next time I review some of the more portable camera-top devices I may get a tin hat to ensure my brain doesn’t end up glowing like an Olympic Torch!

All in all this is a premium product. The full-blown HD system with Receiver is around 27,000 Euro. It is not sold with a service – you have to arrange that yourself. It’s UDP-based, which helps with the operating costs. The price is comparable with the TVUPack, LiveU and IBIS DNMG units, certainly putting all of these vendors’ products in the premium band compared to Teradek’s Bond at around 7000 Euro. Rightly so, Mobile Viewpoint is set to capitalise on the BBC’s selection, and for broadcasters who are going premium there are many great reasons to give the Mobile Viewpoint route serious consideration.

While owning these tools is still prohibitive for many folks in the Streaming Media sector, it is clear that hire business will emerge soon, and so while clients for this type of product will remain broadcast, security, and emergency service focussed, I expect to see more and more streaming workflows turn to cellmuxes as a quick, flexible, and disruptive alternative to satellite over the next few years.

I’m keen to hear feedback on the evolving benchmarking process and very keen to run other vendors products through this process, so please reach out if you think we could add tests and if you have kit to test.

With thanks to Michel, Hedley, Michael, and Tim.