@johncutlefish's blog

Check out Iterate.fm, my podcast with friend and coworker Tareq

5 Simple Questions to Drive Validated Learning

Published: July 23, 2016

b'\n\n\nHere is a simple model that frames product development as an effort to trigger a change in user behavior. It is accessible and not too jargon-laden. It can be used to describe things on the highest level — a whole business, for example — or a specific, low-level design change.

1 gsDRyfZEkjvaDzOJJTgGeA

Whose behavior will change because of the work?

Our product development efforts target people who share some common traits and characteristics. What are those traits and characteristics, and who are the people that meet those traits and characteristics?

Example: Customer Service Reps (CSRs) at technology savvy companies and their Customers.

What is their current behavior?

What pattern of behavior are we hoping to change? This behavior can be extremely specific — like a particular navigation flow or interaction — or it can be very broad like purchasing habits, lifestyle choices, or major life decisions. How often does it happen? How ingrained is the behavior? How does the Target benefit from the existing behavior, or at a minimum put up with it?

Example: Customers currently submit cases via an online form, and then wait up to 24hrs for a reply. The CSRs currently monitor a case queue and communicate with Customers using email and on rare occasions, the phone. The VP of CS believes that opening up a direct channel will overwhelm the team and may not provide them flexibility in responding to case volume.

How will their behavior change?

Something will change as a result of our intervention. Will the user start or stop doing a particular activity? Will they do something more or less often? Will their outlook change? Will they give you more money and your competitors less money? Will they use your impressive new search feature instead of spending time navigating the tab structure?

Example: Customers will use a communication channel “in-app” that resembles a chat interface instead of using the online case submission forms. They’ll feel comfortable interacting with a rep via chat. CSRs will respond to these in-app messages as if they were virtual telephone calls (and do so immediately, personably, and directly).

How will we intervene?

In the example above, the behavior change actually includes the proposed Intervention (in-app chat). The intervention might be a change in design, a change in tactics, or a change in in packaging, messaging, or technology. We’re mixing the pot in some way and hopefully something will happen.

Example: A new online home selling platform triggers a Target who previously would only sell using a broker to take on the task of selling their own house, and to fork over a fixed fee of $2,000. The Intervention is a virtual coach that adapts advice to local and regional requirements for home selling.

How will the new behavior benefit the user/business?

The behavior change must result in some benefit for it to “stick”. Without a significant payoff, you’ll be hard-pressed to persuade someone to change their mindset and habits. Big payoffs can reinforce significant behavior changes (like paying your company a lot more money). The benefit is the return on investment, where the investment is some shift from the status quo.

Example: Customers will enjoy the personal touch. CSRs will deliver faster customer service without the hassle of a phone conversation. The CSR’s employer will have higher customer retention while keeping their CSR budget reasonably small.

Validated Learning

In the spirit of validated learning, we attempt to validate the various assumptions — explicit and implicit — in the framework:

1 YculttC5xLDY8W 9 Zyxow * The Target exists, and can be isolated and targeted consistently

  • The Target’s current behavior exists, and can be measured and understood
  • We can change the Target’s behavior with our Intervention
  • We can measure and understand their new behavior
  • The new behavior triggers a benefit (Outcome) for the Target
  • The Target values the Outcome sufficiently to change their behavior permanently In most cases, just validating one piece of the puzzle (the Outcome for example) is not sufficient. You need the full picture. And it is easy to get bogged down in the weeds — a particular intervention, for example — and lost sight of the major benefits required to really change the ballgame.


Additionally, on a high level we can also use this framework to understand the various types of innovation. Some examples:

  • Discovering new and underserved Targets
  • Address the needs of a large Target with a small behavior change and a small, but proportionately valuable benefit
  • Address the needs of a narrow Target with a large behavior change and large benefit
  • Figure out how to create a better outcome without significantly changing someone’s behavior
  • Figure out how to trigger significant behavioral changes that result in a big benefit Innovation changes one or more parts of the equation in new and novel ways.

And that’s that. Talk these questions over with your team, and prioritize your product development efforts to successively validate the various parts of the framework.