@johncutlefish's blog

Check out Iterate.fm, my podcast with friend and coworker Tareq

“Let’s Run an Experiment!”

Published: January 28, 2018

Some quick thoughts on continuous improvement experiments.


  • Encourage divergence when “casting the net” for opportunities
  • Resist imposing one “version” of what better (and progress) looks like
  • Map the relationships between opportunities. Be skeptical of “one root cause”
  • Converge on common needs, salience, momentum, and potential impact
  • Consider doing nothing. “What would happen here if we did nothing?”


  • Don’t abuse the word “experiment” to force and manipulate
  • Engage the whole team in brainstorming possible experiments
  • Create a backlog of experiments, but limit experiments in progress
  • Engage the whole team in the design of the experiment
  • Remind people that “failed” experiments offer valuable learning
  • Openly discuss how the experiment might backfire. Invite skepticism
  • Discuss steps to guarantee the experiment is safe
  • Timebox the experiment. Use constraints creatively
  • Make sure you’ll have access to data…don’t hoard data
  • Commit to reflect on the experiment regularly
  • Amplify what works. Dampen what doesn’t
  • Rinse and repeat