@johncutlefish's blog

I am currently writing weekly here and have all my 2020 posts here.

Do You Ask For Story-Level Estimates?

Published: February 16, 2017

Given the situation below, do you ask the team to estimate individual stories? If so, why might you need story-level estimates? How would these story-level estimates benefit you? If not, why are you comfortable not having story-level estimates?

This is not a trick question. It could go either way. I’d love your feedback in the comments.

The (Fictional) Situation

  • B2B SaaS vertical solutions provider for SMBs
  • Latest NPS survey shows a high concentration of “detractors” complaining about the lack of a particular capability. It is costing customers hours of manual work per month, and they are pissed. There are a range of solutions that may fulfill the need, but no clear winner
  • Detailed customer exit interviews reveal that a growing percentage of customers choosing not to renew their annual contracts are doing so because of the lack of this capability
  • Existing customer interviews indicate that even longtime customers (who were once “promoters”) are considering leaving for a competitor if this issue is not addressed. Historical data on NPS surveys for this product show that a certain % of detractors will consistently churn
  • Sales is losing deals because this capability is missing, as evidenced by demo call transcripts and “closed-lost” interviews
  • The team believes ~$4,200,000 in existing recurring annual revenue is at risk (around 350 out of 10,000 customers), as well $69,230 per week in lost new ARR for new deals
  • The 350 customers are expected to leave over the next 6 months, so for every week of delay we estimate it will cost the company — between existing churned customers and lost new deals — $230,768 per week in lost revenue
  • The product development team (one of 20 such teams at the company) consists of 4 engineers, 1 UX, 1 PM, and 1 QA, and costs ~$20,000 every week
  • Team is able to ship code at will (multiple times weekly, if required)
  • Team has been stable for >6 months and has experience delivering value to the persona in question (and ownership/familiarity with this area of the code). The team is savvy when it comes to decomposing stories, experimenting, validating assumptions,pivoting when required, and delivering
  • Supporting this type of feature also involves 1) training the support team $30,000, 2) updating marketing collateral $15,000, 3) training the sales team $30,000, 4) free customer training sessions $10,000, and 5) a couple of expensive C-level meetings $10,000. $95,000 in total
  • No extra infrastructure is required to support the feature
  • We ask a senior interaction designer, UX researcher, and lead engineer to take stock of the whole effort. Together, these three team members have a combined 50 years of experience with similar domains. They have a chat for fifteen minutes. The ID and researcher see some UX complexity and suggests the team might want to try a couple versions of the flow with real customers and a working version. The lead engineer doesn’t expect any surprises unless they hit a known limitation in the current API . Both agree that they would be surprised if it took >5 months to successful iterate towards a viable solution (less than 10% chance). There are some potential solutions that could take as little as 3–4w. So somewhere between 4w and 5 months
  • The product development team participates in a “workshop” including 1) a story-mapping exercise, 2) interviewing visting customers and a review of the qualitative and quantitative data 3) a design studio activity exploring different UX options, and 3) a review of the existing API to assess potential risks. This yields a couple possible directions, but again there is no clear winner