File this post under Listicles John Creates to Keep His Own Thinking Straight

An actual set of notes … my spelling is atrociousWhat is it like to drop into organization and take stock of the team?

I was asked recently to describe what was going on in my head when I meet with teams for the first time. There are scorecards and maturity models for all this, but I found myself talking in terms of themes and areas of inquiry. Below I’ve taken a stab at listing what I look for when I’m doing a health-check for a new team. Hopefully this is of some use to internal coaches, analysts, consultants, VCs, or individuals vetting prospective employer.

Important! It is very tempting to let your biases run rampant here. Keep reminding yourself to focus on individual, team, customer, and organization needs. What must this team excel at given their situation?

1. Decisions. What is their product development decision track-record. A perfect track-record is not what we’re looking for. How quickly do they make reasonably good decisions? Do they flounder until someone makes a political breakthrough and advances their specific agenda? Or are decent decisions delivered more regularly? Equally important, look for the process by which the team measures and reflects on the quality of their decisions.
2. Information Flow. How does information flow from the market and customers to the builders and designers on the front-lines? Where is there signal loss? Where are there switches and filters? Where is the signal amplified?
3. Continuous Improvement. What is their approach to continuous improvement? How is it directed and controlled? Who participates? Can the team point to tangible examples of improvement? Look for efforts to take credit for improvement efforts. Struggle over ownership of change efforts is often the blocker to meaningful change.
4. Internal Feedback Loops. Thinking beyond the market/customer feedback loop, describe how feedback moves internally … up, down, and across. Is it funneled, and if so by whom? Who “knows everything” and why? This is related to continuous improvement, but worth mapping out to compliment that discussion.
5. Handoffs. Spend time observing handoffs. These can be incredibly informative. How does an idea get delivered? Who is waiting for what? Are sprints staggered? Focus on the I/O of the organization and the various functional roles. Map these out. Are these handoffs a function of team needs, or the needs of the organization to control work?
6. Utilization Rates. Is utilization very high? Scan for examples of “keeping people busy” and chasing high utilization. Also look for mechanisms that preserve slack. Couple this with attitudes towards multi-tasking and tackling concurrent projects. There is no right/wrong here (there are economic reasons to tackle multiple projects), but it pays to pay attention.
7. Flow. I’m a big believer in the concept of flow. Mostly because it’s something people everywhere can relate to. You’re suitably challenged, you’re in the zone, and you exude a calm, productive energy. You sense your impact. Are individuals and teams experiencing a sense of flow? How often? If not, what happens instead? Is there high drama, a lot of start and stop, or a sense of being “stuck”?
8. Debt Work-Down. How does the organization monitor and work down its various areas of debt (learning, UX, tech, cultural, etc.) ? Is there shared understanding around the costs/rewards? Does the team routinely work down debt, including the removal of unused features? How are these costs “accounted” for?
9. Roadmap. Ask to see the product roadmap, and discuss how it is used as an artifact across the organization. Is it a list of features? How are the goals described? Is it shared broadly? Is it a living document? Why, and why not? Explore how other roadmap users (from the front-lines to the leadership team) view the roadmap process. Are people getting value out of the roadmap?
10. Data/Evidence. How is qualitative and quantitative data used to drive decisions? Is data a regular part of the decisions making process, or is the approach more “ad hoc”? What is on the monitors in the hallway? Where do the data-silos exist, and do people care?
11. Mentorship/Coaching. What is the approach to mentorship and coaching? Approach this from a broad perspective (most efforts are not formal). Do team members attribute their personal and professional development to the organization and/or people in the organization? How do people grow? Where does someone go if the blocker is their boss?
12. Ideas vs. Problems. Is this an idea oriented team or a problem focused team? Check for mentions of the best idea winning, the need to advocate strongly for your ideas, or celebrating great ideas. Contrast this with a problem focused frame, where the team relates strongly to solving a particular problem for a customer/user (or the organization). One isn’t better than another, but this will have a strong impact on how the org operates.
13. Features vs. Outcomes. Somewhat related … is the team feature-centric or outcome/mission centric? Pay close attention to how/when teams celebrate, and what they celebrate. Look at user stories and backlog items. Listen during meetings. Is a project “done” when it ships?
14. Collaboration. How does collaboration happen in the day-to-day? Spend time observing people work together. Is the mood forced and artificial, or voluntary and natural? Contrast this with the official narrative on how work gets done.
15. Cross-Functional Interactions. Focus specifically on cross-functional interactions. Where is this easy, and where is it hard? Where do buffers exist? Who initiates these interactions? Is it driven by process, or by local need? Is the office organized to encourage the cross pollination of ideas?
16. Work Division. How is work divided among teams and individuals? Refer to the org chart, and contrast this with what you see on the ground. Are there functional silos? Component teams? Is Conway’s Law in effect? Who makes the decisions on how to divvy the challenge? Are these structures morphable and adaptable to address present needs? Do teams have some say in what they work on?
17. Safety and Trust. Is there sufficient safety and trust for teams to “get real”? Where can this happen and where does it happen? Where are the natural boundaries to safety? Could a front-line engineer have a frank discussion with the CEO? What would happen?
18. Cadenced Meetings. Do an inventory of cadenced meetings. Who is invited? Who actually participates? How does the output of the meeting catalyze action elsewhere?
19. Releasing Product. Can the team release product without fear? And is there a devops mindset? Figure out what is actually involved for something to get into the hands of customers. Who must approve? Who must touch it? How does the team know if something is amiss?
20. This vs. That. Listen for how the team describes the various dichotomies / tensions in their work. Is it speed vs. quality? Learning vs. shipping? Proud vs. shipped? These narratives can tell you a great deal about the forces at play
21. Core Relationships. The organization will typically mimic the relationships of a couple core individuals (often leaders, sometimes founders, etc.) Look closely at these relationships. Are there overlapping strengths? Productive tensions? Trust?
22. Safe To Fail. How often does the team fail, openly acknowledge that failure, and indicate the ability to learn from that failure? Is their approach too safe? Could they be taking more risks? We are looking for a healthy amount of failure, otherwise there’s likely not enough learning happening
23. Messy Middle. Look closely at the “messy middle” of the organization that sits between leadership, and the individuals doing the work. What happens there? What is the value add? Is this group open to change, or intent on controlling it?
24. Me vs. We vs. Them. Focus on how words are used. Listen for I, Me, Us, We, They, or Them. How does that manifest in how individuals and teams interact?
25. Craft Communities. Are there informal communities of craft (and practice) ? You’re looking here for the ability for the various functional groups in the org to support themselves vs. needing more formal structures.
26. Expert Status. What is this organization great at? By great I mean true, world class experts. Is that known and celebrated? Is it the “right” thing to excel at?
27. Walls. What is on the walls? You’d be amazed at what you can learn by just looking at the walls. Are the whiteboards outdated? How about stickies? Kanban boards?
28. Consistency. How is consistency maintained? This is an interesting one. There are ways to maintain consistency that involve a lot of command and control. For example, you can set up a design review board and approve designs individually. Conversely, you can also maintain design consistency with a living pattern library and communities of interest. That’s one example. You can expand this to other areas including quality efforts, hiring, etc.
29. Celebration. Pay attention to how the organization celebrates its wins and learns from its mistakes. Do teams get credit? Individuals? Is work viewed as a slog, or is there a natural ebb and flow?
30. Narratives. I’m amazed by how many teams are incapable of sharing their foundational stories. What has brought them to this point? When did they slay the last dragon? Dig for stories as they form the foundation. Hope this helps others as they analyze a product development team.