Show me the data.
A few years ago I attended a conference by the Nielsen Norman Group (great stuff!). As one should at a conference, I got a new sticker for my computer. This one, in big text, saying: “Show me the data”. I guess the organizers hoped that a sticker wasn’t what gave me the most… But try it out, see what happens when that becomes your go-to phrase and mindset.
This article is an introduction to data thinking and how you can take your product to the next level. In the following you will get an overview of:
-
What data thinking is and how you can use data in your decision process.
-
What a measurable goal is and why that is important.
-
What data is and how to collect it.
But, first thing first. Before scratching the surface of data thinking, we need to align on what we do when working with software:
When working in a product team, regardless of role, we’re not delivering software. We’re delivering value to the end-users through software.
Why is this important? Well, hopefully, that wasn’t news to anyone. But to be able to build the best products we must understand the purpose of what we’re doing and the results of every decision – and that we never can afford to base any decision on assumptions or gut feeling.
Data Thinking
Data thinking is a mindset and, a (if not the) way to approach product development. It’s about keeping data and (measurable) goals present at all stages of your decision-process. During the define, build, and evolve phases.
By doing so you can identify and eliminate assumptions faster, leading to more informed and knowledge-based decisions. By working with measurable goals you’ll have a clear and shared definition of what you’re about to build, why, and more importantly, how it’s expected to perform. This will give you the possibility to verify, learn, and improve, continuously.
Measurable Goals
From our point of view, measurable goals are the key to succeeding. Without measurable goals, you’ll have a very hard time to follow up on your releases and to interpret the data you’ve collected. So, what is a measurable goal?
Measurable goals are absolute goals that can be expressed in numbers. Say that we’re making a redesign of a product’s sign-up page. A none data-centric goal could look something like this:
”By making this new design we should get an increase in the numbers of users signing up”.
But what then? How do we follow up and how do we know if we improved it enough? Is a 10% increase good enough or is a 1 user increase good enough? If we instead state our goal as:
”By making this new design we should get a 15% increase in the numbers of users signing up”.
Then it is much easier for us to determine if we’ve reached our goal or if we should take further actions.
The hard part here is putting a number on it. If you’re redesigning an existing thing it’s slightly easier because you might have some data to build upon. It’s when you’re building a new feature or a new product things can be harder when you have less to compare to. It’s usually now things are getting scary for many teams.
Because now you have to start to think about your user group, user value, and the business side of things. Why do you want to do what you do and why is it worth the investment? That should be the starting point of your goal and what should be reflected in the number. It’s easy to be very money focused in these discussions (which of course is the heart of every business). But remember that there are many paths leading to improved business and return on investment.
What is data and how to collect it?
A very central question is what we mean when we talk about data. Usually, people think of big databases, user data, and user-generated events. And yes, of course, that is data. But it doesn’t have to be that complicated.
Data and data-collection could also mean manually gathering information from a user test. Or keeping track of the number of external questions your team gets during a week. As soon as it’s information that you can compare and track over time it’s data. It’s also worth mentioning that even feelings and experience can be measured. You just have to break it down and define what it would mean in the context of your product and/or team.
So, you don’t have to have a big dataset with hundreds and thousands of data points and you don’t have to have a big fancy tech solution for gathering data. It can be as simple as a piece of paper. It all comes down to what you need answers to and what you want to achieve. In some cases, it makes sense to build a full data pipeline, custom dashboards, and data interpretation tools. In other cases, it doesn’t.
Key takeaways
-
When making a decision, discussing a story, feature, project – ask yourself the following questions: Do we really have data on this? If not, on what ground do we base our decision? And even before that, what are our measurable goals? What have we defined as success? How can we verify what we’re about to build?
-
A dataset can be small, and be manually collected, and still be data.
-
Define the goal(s) and put a number to your expectation – for every feature, story, and project.