Using data to make smarter learning decisions

The starting point for tackling data L&D’s challenges

Data is the hot topic for today’s learning leaders. Nine out of ten learning leaders are looking to data to help them make smarter decisions¹. Personalised, adaptive learning, artificial intelligence, and learning analytics topped Donald H Taylor’s Global sentiment survey² at the beginning of the year.

Fuelled by award-winning applications of AI³, and in-depth conversations about the user experience and demonstrating impact, the data trend has continued to build momentum throughout 2019.

However, whilst data might be the new black for L&D, not everyone is comfortable wearing it!

L&D’s data challenges

In the run-up to our joint webinar with Valamis on the 5th of December, we conducted a poll to explore the challenges learning professionals were experiencing in using data to make smarter decisions. L&D people from a wide range of industries and countries identified 109 challenges that they are facing with data as they move into 2020. Figure 1 shows that the challenges fall into five main categories:

^{Figure 1 L&D Data Challenges (n=107)}

A significant issue is the confidence of L&D to work with data, with 26% of challenges linked to either basic digital literacy or to more advanced data analytics expertise. The lack of data analytics skills was not a surprise, as it has also been widely reported by the Learning and Performance Institute⁴ and Towards Maturity⁵. But it was interesting that 17% of the confidence issues went right back to basics, asking fundamental questions about data principles, such as: What does data even mean for us? Where do we start? How do we add value with data?

Overall, the challenge of accessing data was the biggest concern—both collecting new, relevant data and accessing other data sources already in use. Technology, in theory, should promote access, but 10% of the issues raised were directly related to the tools, finding the right technology to capture and analyse data, getting those tools to work with other platforms, and, in some cases, L&D bemoaning the lack of actionable data from existing platforms.

The application of data to practical problems represented a challenge for 1 in 5 respondents, with 11% asking questions, such as how data we have can help us personalise learning, make smarter decisions, demonstrate impact and more. Nine percent expressed concern about the relevance of the available data to address the real challenges ahead.

Lack of support stems from the lack of buy-in from the wider business world which, in turn, contributes to the lack of resources (both time and money) to fully exploit the opportunity that data can provide to L&D. It was widely recognised that data from multiple sources is needed for L&D to be able to make smarter decisions that ultimately support business performance. Realising this requires a collaboration with business leaders and data experts, which is still a challenge.

The final concern raised within this sample was linked to the basic issue of trust. It was not only about the ethics and permissions involved in gathering useful data, but also the hidden assumptions—from both business leaders and learning leaders—about what good data looks like and the context in which is it collected.

Where to start?

As L&D leaders, we are surrounded by potential sources of data from our systems, our companies’ management information and external research. We can also surface data from the reflections of our learners and the questions we ask of them and our stakeholders. These data points are just individual information sources. As such, they can be meaningless when considered in isolation. They can be misleading, misused and sometimes downright dangerous when taken out of context. The sheer volume of data sources can be confusing and frustrating, but that is not our greatest challenge. Tom Davenport, author of the book Competing on Analytics: The New Science of Winning, says that “The biggest problem in the analysis process is having no idea what you are looking for in the data.”

Data only starts to make sense to us when we start to ask smart questions of ourselves and our services.

For all of these challenges, the best place to start is not with the data or developing a new set of data-analyst skills, but rather with our own curiosity. What do we want to prove, improve or even disprove? We don’t need a degree in statistical analysis to start asking questions. but we do need a curious and critical mind.

Data – benefits and challenges

Data is one of the true benefits of this ubiquitous technology. L&D leaders are flooded with new facts and statistics, all of which are potential sources of insight that can be used to adapt and improve. It’s not surprising that for the first time, learning analytics is at the top of Donald Taylor’s 2020 L&D sentiment survey.

Data also brings its own challenges.

Despite the potential data can encompass, L&D leaders struggle to access the data they need, and they often do not trust the data they do have access to, in addition to not feeling supported by their organizations in gathering and using it. Most importantly, 46 percent of the challenges reported were rooted in lack of confidence to understand and apply data to help solve the problem at hand.¹

Understand the problem

Data can only start to make sense when we apply it in order to solve a problem.

What do we want data to improve? To prove? To disprove? Our own curiosity is the starting point for harnessing data more effectively, and the challenge we have with data often starts with our lack of focus on the problem.

The right questions can help us make sense of the problem. They help us break down our challenges into solvable pieces.

When we have clarified the question we need to ask in order to solve our problems, there are multiple options for finding an answer.

We can follow the crowd – what’s “hot” right now? We can go with our gut instinct – with all its intuition and bias. We can leverage our own experiences – both good and bad. We can seek out independent research on the question – or conduct our own. What’s clear is that data alone is not enough — it needs to be applied to a specific problem. Therefore, we need to shift our focus away from the data and onto the new “e” in learning – “evidence.”

Evidence – the new “E” in learning

Evidence is defined as ‘the available body of facts or information indicating whether a belief or proposition is true or valid.’  Evidence helps us not only make sense of data but also helps us to direct its power.

According to Centre for Evidence-Based Management, ‘’The basic idea of evidence-based practice is that good-quality decisions should be based on a combination of critical thinking and the best available evidence.’’²

They discuss evidence coming from a range of sources –

• Scientific studies – Dr. Will Thalheimer, Patti Shank PhD, and Mirjam Neelam all do an excellent job of curating relevant research for the L&D community.
• Your organization – your learning data, your business data, and your stories and anecdotes all provide a unique perspective to your question.
• Practitioner insight – professional experience and expert knowledge that is constantly honed through reflection and dedication is a valuable evidence source.
• Stakeholder’s values and concerns – the values and concerns of stakeholders provide you with context that then shapes the environment for success.

All of these sources provide one level of insight to the problem at hand. Their strength and power come from combining them into an evidence base to support smarter decisions and more productive conversations.

L&D are faced with many decisions and questions in this rapidly changing, digital world of work. This is why we need to make evidence the new “e” in learning!

Evidence builds courage – we are more confident in making better decisions, more likely to reduce risk, and more willing to be creative and to challenge the status quo.

Evidence also opens new doors – new conversations with leaders, new opportunities to add value, and new relationships to enable change.

What evidence can you start to leverage today?

Citations:

1. L&D’s Relationship with Data
2. GSS19 – Key Results Infographic
3. Travel company TUI won most innovative new learning product for their use of chatbots in Learning Technologies awards 2019
4. LPI capability map
5. Towards Maturity Transformation Journey 2019
6. Using data to make smarter learning decisions – part 1
7. What Is Evidence-Based Management?