How do we know our work is working? How do product designers know their work product is solving the problem it was intended to solve? That’s the kind of question that keeps us up at night.

“It’s an insidious question,” says Kate Rutter, designer, tech junkie, artist, and Principal at Intelleto. In this episode, hosts Sean and Joe chat with Kate about Metrics, but not just any metrics and not just those that only measure performance. Kate says the true power comes from our alignment around metrics as a very tangible element that people can get behind. “It gets really exciting when you…start to see metrics as human behaviors with your products stated in numerical terms.”

Choosing the Right Metric

Whether to measure is a much simpler question than what to measure. Some metrics – e.g., vanity metrics – generate seemingly impressive data that are great to rally around. They may be easy to gather and make stakeholders and investors feel good about themselves. But take caution: danger lurks when you make high-stakes business decisions around the wrong metrics.

The key is to find the metric that fits your business, Kate says. “The right metric for you is “the one that is as unique as your business, specific enough that, just by hearing it, people are able to tell you what your product does.”

At the product level, the right metric compares a product’s performance to its purpose within the lives of your customers, Kate says. That kind of measure becomes actionable not only in tracking the product’s performance over time, but incorporating it as part of the designers’ product enhancement playbook. Now they can translate hard data that represents users’ behaviors into UX enhancements and features that solve real customer problems. Actionable metrics make product managers feel good because they can measure the use of the product as it solves customer problems, and then improve it over time.

Closing the Gap between Metrics and Behaviors

The heavy lifting, Kate reveals, comes in closing the gap between designers’ ability to integrate metrics (and the goals those metrics represent) into how we shape our actual product behaviors – that is, the features we decide to work on and the purposes of those features. “That is what I see is a real gap,” she adds.

As researchers strategically peel away the layers of user behavior, we help product managers examine the underlying performance of products and their features. The purpose of metrics is twofold: the first is to collect data about how products perform. The true value comes when we apply that knowledge to understand more precisely how end users behave with the features and products we create. This virtuous cycle continues when we fine tune not only the features users need, but the metrics we use to gather even more data about their behavior.

“Our product has to work for the people,” Kate says. “And when we measure how it works for the people, our products can improve much more dramatically.

Kate offers a powerful example of the interaction between feature enhancement and metrics based on user needs and user behaviors. Starting with the concept that there’s “this thing that people need to be doing with your product that you just have to measure,” she describes a continuum of inquiry and user research using a consumer mobile app for task sharing.

Applying a scientist’s logic, Kate guides us through the process where passion and curiosity come together to sharpen focus. “This level of questioning really helps us forward, building the right thing for the right reasons,” Kate says.

This is tough sledding, and the work is hard, Kate points out. “But it’s only this level of user research – and user research is the place where all the best metrics come from – when you really are able to nail that core action…and measure it and adjust it and change interactions. That’s an extraordinary skill; that’s kind of the Holy Grail.”

To learn more from Kate’s conversation with Sean and Joe, especially Kate’s analysis of Jesse James Garrett’s “more statement,” check out ITX Product Momentum Podcast – Episode 9: Finding the Right Metrics.

References:

Jesse James Garrett, The Elements of User Experience.

Kate Rutter and Laura Klein, Users Know (a podcast).

Sean Flaherty, Customer Loyalty Is Not a Transaction.