L&D Data - Behind the Scenes Part 1

As we enter 2022, we’re excited about all the research that we’ll get to do this year – the questions we’ll ask, the data we’ll analyze and the reports we’ll write. And where better to start than with our Learner Intelligence project?

In a three part series, we’re going to take you on a brief journey from inception to insights – offering a behind-the-scenes look at what goes into our research from the researchers themselves, here, at Mind Tools. And why do we want to do that? Because we believe that research should be open and honest, and that by opening the door to our practices and processes, L&D professionals around the world will have confidence in the work that we do. Not only that, but we hope we can motivate other likeminded researchers to replicate our findings and join us in our mission to build a collaborative research culture.
 

The brief: Learner Intelligence 

Learner Intelligence is a longitudinal study run by the research team here at Mind Tools with the aim of unpacking what makes learning in the workplace impactful. So far, we’ve collected responses from more than 69,000 employees over a 7-year period and we’re currently preparing to publish our 5th report in early 2022.

Figure 1. Take a look at our Learner Intelligence Report from 2020

Year-on-year, we’ve collected data from organizations that we work with on a regular basis. Once aggregated, this data has given us an insider’s perspective on what L&D looks like in the workplace and, more importantly, how this relates to employees’ attitudes and beliefs.

Insights of these kind are crucial to all L&D stakeholders, whether you are a leader seeking to improve your organization’s L&D strategy or an L&D professional designing new and engaging tools for learners. This time round, however, we’ve decided to team up with YouGov to collect our sample and explore some of the questions we’ve been thinking about over the course of a pretty turbulent year.
 

So, let’s get started! 

As this is the first blog in the series, it makes sense to start at the beginning. Our approach to research is to begin broad and consider some of the bigger questions we’d like to address. As you can imagine, the process is never as smooth as we plan it to be, but that’s what keeps things interesting!

Figure 2: Research expectation vs. reality

As this is our 5th Learner Intelligence report, we’re not starting with a blank slate. Instead, we have a bank of questions from earlier surveys to reconsider (are they still relevant?), knowledge about what has or hasn’t worked in the past (could questions be clearer?) and insights to follow up with (last year, nearly 40% of learners reported that a lack of time is what restricts them from learning at work, but do they feel the same this side of the pandemic?).

It was at this point that we considered other factors like how we would analyze the data, the broader impact of this work for the L&D community and what demographic information we would need to collect. For example, as part of this year’s survey we’d like to know if an organization’s size has an impact on the number and types of barriers learners face at work, and so it’s crucial we gather this information in addition to all the obvious stuff (e.g., respondent’s age, gender, and location).

However, with a new year also comes new perspectives. We’re in a different place today than we were in 2020 and, indeed, 2021. Working and learning from home is up 37% and looks like it’s here to stay, at least for the time being (Office for National Statistics, 2021). Mind Tools’ Learning Performance Benchmark also shows that organizations are no longer just “staying afloat” like they were at the start of the pandemic but are once again making time to think about the future of their learning strategies. That’s why our research starts with a review of what’s been going on in the world from other L&D professionals and academics.

For us, the cumulation of this work – reconsidering earlier surveys and reviewing the research literature – is brought together with the help of a Miro board. Miro boards give us the space to visualize the research process, collaborate with colleagues and identify gaps in our work – the output of which can be viewed here (enjoy!).  

Figure 3: A snapshot of our Miro board

Research reminders 

Having drafted our questions, we handed them over to our friends at YouGov, where we were reminded of a valuable lesson: be brave in pushing back. When you’ve drafted a survey that is grounded in research and thought carefully about your hypotheses, you can (and should) have confidence in your decisions. This is an important one, because sometimes your decisions are challenged.

YouGov were understandably keen to ensure that the questions we were asking were as clear as they could be, but this didn’t always work for us. Sometimes making questions “clearer” actually changed the questions or had the potential to influence how respondents answered them. For instance, we started the survey by asking:
 

What do you think learning is? 

This question was left intentionally open to interpretation because that’s exactly the information that we’d like to capture – what people think learning is, free from contamination.  By asking this question, our aim is to identify a full range of explanations, to code them based on their key themes and to explore if and how they impact self-reported learning behaviours.

While YouGov suggested that we include some context (e.g., “What do you think learning at work is?”), we pushed back because we had confidence in our decision process.
 

The waiting game… 

The survey is finished, and data collection has begun – this is the hard part for impatient researchers like us – but we’re thrilled to get to this point. Sit tight for blog #2 where we’ll take you on a tour of the analysis process. If you have questions about our research, please Tweet us at @MTInsights – we’d love to hear from you! 

Now, it’s time for a brew 'cause there’s not much else to do!