Ep. 12 Why Your L&D Data Strategy Is Failing

Mastering the "Understand" Phase in Product Design for Learning with Tom McDowallEpisode SummaryIn this episode of the Product Design for Learning podcast, host Greg Arthur sits down with Tom McDowall, Chair of The Learning Network, to dive deep into the "Understand" phase of the product design process. Tom explains why this phase is critical, sharing practical advice and strategies on assumption mapping, stakeholder engagement, and data analysis. Listeners gain insights into balancing fact and assumption, managing risk, and ensuring that learning solutions genuinely address performance challenges rather than merely looking engaging.Guest ProfileTom McDowall is the Chair of The Learning Network and a seasoned expert in learning design and product strategy. Known for his candid and pragmatic approach, Tom brings years of experience in tackling real-world performance issues through thorough research and data-driven methodologies. With a reputation for challenging the status quo, his insights help organisations move beyond superficial solutions to create impactful learning experiences.Key Take-AwaysHow would you summarise the "Understand" phase in 60 seconds?Tom stresses that the "Understand" phase is the most crucial step in the design process. It sets the foundation for all subsequent phases, and if done incorrectly, nothing else matters. He highlights that this stage involves rigorously examining assumptions and real facts, and the phase must be approached without shortcuts to mitigate unnecessary risk.What is assumption mapping and why is it important?Assumption mapping involves breaking down a client’s request into facts and assumptions. Tom uses this tool to clarify which parts of the brief are evidence-based and which are speculative. This clarity prevents the design process from veering off course and helps in making informed decisions that address genuine business risks.How do you deal with stakeholder pushback?Tom explains that pushback often comes in the form of pressure over deadlines and budgets. He advises identifying who truly controls these aspects (budget holders, timeline setters, and key decision-makers) to build alliances. Engaging those who may even disagree initially can provide a more realistic picture of the workplace and improve the overall output.How do you balance individual work with team collaboration during the "Understand" phase?Tom emphasises the value of interacting directly with stakeholders and end users. While solo work can lead to faster ideas, it risks missing crucial feedback. He recommends a mix of interviews, roundtable discussions, and group activities to challenge assumptions and enhance understanding, ensuring the process is inclusive and reflective of real-world conditions.What challenges and joys do you experience during this phase?Tom loves the early stages of gathering data through interviews and fieldwork but admits that analysing the data can be a tedious task. He suggests that delegating data analysis to someone who enjoys it, or using technological tools to assist, can help maintain momentum without compromising the quality of insights.How do you ensure your analysis communicates value?Effective data visualisation is critical. Tom argues that good visualisations should tell a story that stakeholders can understand at a glance—not just a collection of graphs. By clearly demonstrating the impact of potential risks and the benefits of informed decisions, you can secure stakeholder buy-in and move the project forward.What advice do you have for someone struggling with the "Understand" phase?Tom advises not to stress about perfection; incremental improvement is key. Start by distinguishing between facts and assumptions and use that as a basis for further exploration. Even if you only conduct a few quick interviews, share your findings and adjust based on feedback. The goal is to build a robust foundation that informs the rest of the project.Chapters and Time Stamps[00:01] – Introduction and Guest WelcomeGreg Arthur introduces Tom McDowall, highlighting his role as Chair of The Learning Network and teasing the focus on the "Understand" phase of the product design process.[01:00] – The Critical Nature of the "Understand" PhaseTom summarises the importance of the "Understand" phase in under 60 seconds, stressing that a solid understanding is essential to prevent costly risks and errors later in the process.[02:18] – Managing Predefined Understand Phases and RiskDiscussion on how to handle clients who come in with preset ideas. Tom explains his approach to risk management by questioning assumptions and reinforcing the need for proper data gathering.[04:04] – Navigating Stakeholder PushbackTom details common pushbacks from stakeholders—such as tight deadlines and budget constraints—and how he leverages direct conversations with key decision-makers to clarify project objectives.[06:16] – Starting the "Understand" Phase: Assumption MappingAn in-depth look at assumption mapping: its purpose, the process, and how it helps to differentiate between facts and assumptions early in the project.[07:42] – The Role of Data and Collaboration in the Understand PhaseTom discusses the balance between working solo and collaborating with stakeholders, highlighting the risks of isolation and the benefits of engaging multiple perspectives.[10:11] – Addressing the Love-Hate Relationship with Data AnalysisTom reveals his personal challenges with data analysis and the strategies he uses to overcome them, including hiring specialists and leveraging technology.[16:56] – Visualising Data for Impactful CommunicationA discussion on the power of data visualisation and how effective visuals can translate complex data into actionable insights for stakeholders.[22:00] – Identifying Key Stakeholders to InvolveTom explains the importance of involving the right stakeholders—especially those who are critical or even sceptical—to obtain an accurate representation of the workplace environment.[28:00] – Measuring L&D Success Beyond Popularity MetricsThe conversation shifts to performance metrics, emphasising that true success is measured by performance improvement rather than superficial popularity scores.[40:21] – Debunking the Myth of Short Attention SpansA debate on whether learning content must be short and flashy, with Tom arguing for the necessity of deeper engagement and meaningful content to drive performance improvements.[47:08] – Historical Perspectives on Performance ImprovementTom reflects on lessons from past decades, particularly from Thomas Gilbert's Human Competence, and discusses the need to focus on environmental fixes rather than over-relying on training.[52:39] – Final Advice for Tackling the "Understand" PhaseTom offers practical tips for those struggling with the "Understand" phase, recommending incremental progress, careful assumption mapping, and continuous stakeholder engagement.[56:15] – Plugs and Closing ThoughtsTom invites listeners to connect with him on LinkedIn and check out his YouTube channel, Instructional Design Tips, for more insights on L&D and product design.About the PodcastProduct Design for Learning is a podcast that explores how product design principles can transform learning and development. Hosted by Greg Arthur, the podcast features deep-dive discussions with industry experts, practical insights, and strategic advice to help organisations create effective, human-centred learning solutions. Subscribe for regular episodes that blend cutting-edge product design with the latest trends in L&D.