The Visualist Blog

Be Honest with Your Customers and They’ll Be Honest with You

Empirical Thinking -Or- Part 2 of Honesty with Customers (and Yourself)

I recently came across a marketing brochure from a competitor that outlines ‘facts’ about their offering and our own. The ‘facts’ turned out to be nothing more than marketing fantasy. Perhaps it’s my academic background, but unsubstantiated facts almost make me break out in a rash. Later in this article, I’ll outline the tests I ran to show their claims are wrong. But I asked myself, why don’t we do something similar? It also made me ask, what role should marketing play at our company?

I would argue that a successful marketing team focuses on informing the market about your company’s vision (why do you exist?), your products and how they are supposed to solve problems, and ultimately gain a seat at the table when users are making purchasing decisions. Marketing approaches that are misleading or are based on hyperbole, in my experience, simply backfire in the long run. A positive approach to marketing has all kinds of benefits – an informed customer will let you know where your product needs to evolve. You’ll also create an honest dialog between a community of intelligent users and your company, plus you’ll build trust. Also, for all the hype about data-driven decision making – misinforming your main source of data leads to poor decisions over the long run. This is particularly true for the AV market, which is full of very smart and technical consultants, as well as resellers who expect (and appreciate) the honest truth from product suppliers.

So, back to that brochure. It’s a comparison of Mersive Solstice, Barco Clickshare, and the Crestron AirMedia 2 put out by Crestron’s marketing team. No surprise – all quantifiable measures in the brochure put AirMedia slightly ahead of Solstice. Wireless sharing is a compelling product category for a number of reasons, but it’s a new space, and misleading customers about the facts does everyone a disservice. With that said, I took it upon myself to look empirically at Airmedia 2 versus Solstice – hopefully with a careful attention to experimental detail to get an unbiased view. Read on to see what I found..

I decided to test two aspects of the products, bandwidth and frame rate. In this blog post I’ll focus on the results for bandwidth – I’ll cover frame rate results in a follow-up article.

Why does bandwidth matter? Bandwidth is the amount of data that a network needs to transport to support the product over time. This is a limited resource that most enterprises have spent millions of dollars on to provide to their users who are attached to the network. In essence, devices that soak up too much bandwidth are viewed poorly. Crestron realized this is an important metric and claims that they use 1.6 Mbps while Solstice utilizes 2.4 Mbps. The brochure, of course, doesn’t cite how they measured these numbers – but a single number doesn’t tell the whole story.

Bandwidth is a measure of total data transmitted per unit time on the network. It’s impacted by the video’s compression ratio and the rate at which frames are transmitted from the users device to the wireless display in the room. Unless you’re encoding with what’s called a constant bit-rate encoder and throttling your frame rate, bandwidth usage will be variable. I designed an experiment, then, to test each product under three different common use scenarios.

My experiments used the latest AirMedia 2 and Solstice version 3.2.1. In each case the same Windows 10 laptop was used, with a resolution of 1080p. To isolate behavior I connected the Windows 10 device to the wireless streaming device on a standalone network and used Wireshark to monitor network traffic. Bandwidth was measured by isolating any network traffic between the client and the wireless host, measuring total data sent and then dividing by the duration of the test to compute Megabits-per-second.

I ran three common scenarios. In each scenario, I tested both products and made sure the desktop content and user behavior matched across both tests. Here are the three test scenarios, increasing in level of “difficulty” for a wireless sharing product:

Desktop Sharing – Normal Use In this study, I shared the desktop to the display using each wireless sharing product for 5 mins, during which time I edited the same word document while sharing live. At the 5 min mark, I opened PowerPoint and entered slideshow mode, advancing slides exactly every 10 seconds.

Desktop Sharing – Video in an App Window For the next test, I opened a video on a constant loop within a media player that covered ¼ of the desktop. I waited 10 minutes, then minimized the video window for 2 minutes, then restored the video app window on the desktop for an additional 2 minutes.

Desktop Sharing – Full-Screen Video In this study, we looked at how effective each product could encode a full-screen video (in our case a free Avengers Infinity War Trailer) over a 5 minute period. We opened the video within Windows Media Player and maximized it to full screen, then measured bandwidth usage for 5 minutes.

Bandwidth Comparison Results

Bandwidth Comparison Results

For normal business use, Solstice clearly outperforms the AirMedia unit. Why is this? Solstice is likely the only product in the market that in addition to looking carefully at encoding parameters and compression ratios, also monitors the desktop for changes. In many use cases when PowerPoint slides are being shown and discussed, the desktop is effectively idle and no data needs to be sent. Unfortunately, many traditional H.264 pipelines (assuming that’s what AirMedia 2 is using) require a constant send rate of “I-frames” to keep the hardware decoder properly fed. What this means is that Solstice can detect this situation and use very little bandwidth while AirMedia sends a perfectly normal 2.4 Mbps. I’ll point out that this ‘best case scenario’ is already well over the claimed bandwidth of 1.6 Mbps.

What else can we learn here? When full-screen video is triggering both products to send encoded video, Solstice’s compression ratio outperforms AirMedia 2 by more than 2 Mbps. Probably the most interesting result is that we are well under AirMedia 2, even when a small video is being shown on the desktop. I’m not sure how to explain the 7.2 Mbps for AirMedia in this case, but it may be related to an encoder that ends up sending entire frames, without other compression mechanisms, when motion is present.

So why did I go to all this trouble? My readers will know that I’m not one to focus on the negative – but in the case of wireless collaboration, it’s important the bar for performance is set properly. I really do believe that with the end of the video cable we’ll enjoy great changes in the workplace that will allow us to leverage our display landscape to share, communicate, and become more productive. It’s also important that we don’t mislead the market with marketing hype that’ll do more damage than good.

Stay tuned for the other part of this story – frame rate. It’s as important to end users as bandwidth is to the IT/AV infrastructure team.

Leave a Comment

More posts like this

Technology at Scale - The Future of Cities

[rt_reading_time postfix="min read" postfix_singular="min read"]

July, 2021

Read Post

tech+ Podcast - Digital Nomadism: The Office is Anywhere

[rt_reading_time postfix="min read" postfix_singular="min read"]

April, 2021

Read Post

tech+ Podcast:
40 Billion Cameras and Counting

[rt_reading_time postfix="min read" postfix_singular="min read"]

March, 2021

Read Post