Putting your assumptions to the test

By Francis Moran

In our last post, we looked at Screenreach Interactive’s recent inroads in the radio and television industries, including its appearance on Popular U.K. television program The Gadget Show at Radio Festival, Europe’s top radio industry event, and its new “experience” for long-running U.K. current affairs program Dispatches.

Making a splash at major industry events and with high profile clients demands one thing – a compelling product. But a compelling product can’t be developed in a vacuum; it must address a clear market demand. As we have emphasized time and again, marketing and product development must work together from the get go. To quote guest commentator Ronald Weissman, “Great companies constantly test the market, for validation and feedback.”

The team at Screenreach has taken this to heart. With a new version of the Screach app expected to launch in February, every effort is being made to solicit input from beta testers and prospective users. In this post, we’ll look at how Screenreach approaches the beta testing process, what third-party tools it has found to make life easier and the lessons it continues to learn along the way.

In beta with social media

While the team is reluctant to reveal too much about the new features and functionality that will be included in the new version of Screach, it is safe to say the app has been in a constant state of evolution since its launch in the summer of 2010. Market feedback has been fundamental to the process, with social media emerging as a primary means to attract and engage users.

Screenreach’s blog has proven to be a useful tool and this time around, the lure of winning a MacBook Air is an added incentive. But building a community following on Twitter has been fundamental to mustering a strong and outspoken group of testers. The team has built its following by getting involved in specific industry conversations on Twitter and constantly discussing Screach through the micro-blogging platform.

“There is a need to strike a balance between too few and too many beta testers,” said Screenreach CEO Paul Rawlings. “Realistically, you may have a lot of people signed up for beta testing but experience has shown us that only 25 percent of those will actively download the app and provide feedback – perhaps even less. So from that point of view you need to account for drop off.”

The mobile platform being tested also impacts how large the group can be. At this stage, Screenreach is only testing the iPhone version and Apple limits beta tester groups to 100. Google, on the other hand, imposes no such limitation for the Android.

Finding the forest among the trees

It’s one thing to build a strong community of beta testers who are willing to provide useful feedback and quite another to manage the logistics of all that crowd-sourced input and distil it into meaningful data. For that, Screenreach has found two useful third-party tools.

The first is Test Flight, a free web service that provides developers with an easy means to distribute their beta apps for testing. For the Screenreach team, there just isn’t anything else comparable.

“TestFlight has proved to be a useful tool for us, it has a great dashboard and organizing tools,” said Rawlings.

However, its current version does have one drawback.

“One problem we have experienced with TestFlight is that it only allows you to assign one account to your device which has posed problems when a beta tester has previously registered another account,” said Rawlings. “It means they have difficulty swapping the accounts over – this was something we were already aware of and could help the testers resolve but it’s definitely something you need to be aware of. Anything that inhibits the tester from downloading the app as quickly as they need to obviously isn’t great.”

The other tool is Get Satisfaction, an online platform for brands to host their community of users.

“We looked around at a number of tools that could be used to collate feedback on the new app and we chose Get Satisfaction for a couple of reasons,” said marketing manager Sarah Athey. “First, we really liked the way it allows a user to categorize their posts. It means we can manage feedback based on whether it’s a question, idea or problem. It has useful admin features too, such as a management view which helps you to track and stay on top of all your posts.”

The Screenreach team never really considered handling these kinds of logistics in house.

“Third party apps like Get Satisfaction are good at what they do and make it easier to categorize feedback topics and assign the best team member to respond to a specific question, problem or idea from a tester,” Athey added. “It also creates a forum for testers to compare notes with each other and for the whole team to get involved in these conversations.”

In some instances, however, nothing beats good old fashioned face-to-face contact. While Get Satisfaction has proven effective for engaging in conversations through online forums, the Screenreach team has still found it useful to hold focus groups for feedback on the Screach experience for versions that have been customized for a specific market vertical, such as television programming.

“We’ve also had a couple of testers who have really gone the extra mile and their fresh perspective on Screach has been fantastic, so we’ve thought about inviting them in to meet the team and see the impact their feedback has made,” said Athey. “Some of the testers put a lot of their time and thought into the process and we’re very grateful for that.”

Lessons learned

Despite the benefits of using social media and specialized third-party tools, the team has found that the process can be more complicated than expected, with a host of small details to manage around the release to ensure the install of the beta version is as painless as possible for testers.

“One thing we’d say is make sure you have the full team involved in the process. Everything that comes out of this involves just about every aspect of the organization,” said Rawlings. “It’s also an interesting way to generate new ideas within the team. We’ve always placed great emphasis on every team member being deeply involved in the user and client perception of Screach so this is an important part of that.

“For example, your developer needs to be ready with the technical aspect of things but this has to be correctly communicated to your marketing person for them to translate that message to the user. Then there will be feedback that revolves around design or bugs or product questions in general – you need to have all of the correct people on board to answer these questions.”

One risk the Screenreach team took was opening beta testing during the holidays.

“We took a bit of a risk, but as it turned out, it meant our core testers had free time on their hands to get more actively involved,” said Rawlings. “You need to be aware of things like this.”

The important thing, Rawlings added, is to immerse yourself in the feedback that you receive by asking followup questions to understand the thought process behind why a beta tester is making a specific suggestion.

The learning process never ends. The new version of Screach will soon be pushed out to Android beta testers, which promises to provide a whole new flood of feedback given the growing variety of Android devices that are now available.

“We’re aiming to launch in February so things are very full on at the moment and the beta tester feedback is at the heart of everything we are working on,” said Rawlings.

In our next instalment, we will take a closer look at the launch efforts for the latest version of Screach, provided, of course, that the team’s best-laid plans do not go astray.

This is the fourth article in a continuing monthly series chronicling the growth path of Screenreach Interactive, a startup based in Newcastle upon Tyne in England’s North East. Screenreach’s flagship product, Screach, is an interactive digital media platform that allows users to create real-time, two-way interactive experiences between a smart device (through the Screach app) and any content, on any screen or just within the mobile device itself. We invite your feedback.

Francis Moran
Francis Moran
Francis Moran is principal of Francis Moran & Associates, a consultancy that provides business-to-business technology ventures with the strategic counsel required to make their innovations successful in a highly competitive marketplace. Francis can be reached at [email protected].

Would you recommend this article?


Thanks for taking the time to let us know what you think of this article!
We'd love to hear your opinion about this or any other story you read in our publication.

Jim Love, Chief Content Officer, IT World Canada

Featured Download

Latest Blogs

ITB in your inbox

Our experienced team of journalists and bloggers bring you engaging in-depth interviews, videos and content targeted to IT professionals and line-of-business executives.