Unlike websites and mobile apps, there aren't convenient analytics tools for smart TVs. One of the most important work I am doing at Hisense is to create protocols for smart TV's usability testing and heuristic evaluation.
To get feedbacks at an early stage of product development, I designed a testing method that uses simulators within web-browsers to enable remote, quantitive research. Participants can visit the test from a URL and we will be collecting task time and recording user inputs.
We tested the method on Launcher UI with over 100 participants recruited from Craigslist and social networks. I chose TV's launcher because I've seen very different practices among competitors and through Hisense's own products' evolution. We desire a set of in-lined and proven design principles.
For this test flight, I picked three different designs:
The participants were asked to locate two targets on all designs: One within the screen as launched, and the other requireing scrolling and searching for.
The test gave very interesting results. The design which we liked the least internally actually outperformed others. The users' behaviors also presented a pattern, from which I was able to draw some basic conclusion.
Please contact me for more details.