Big Data Mistakes You’re Making and How to Fix It

Over the past few years, research has shown that while learning analytics and data are seen as a vital area of focus for learning and growth, learning professionals do not trust their data skills and see big data challenges ahead as they respond to business calls to be more data-driven. Where are L&D’s possible pitfalls, and how can they prevent them?

L&D also seems to be suffering from low self-esteem when it comes to Big Data and its challenges. “every department in the organization has better data than you”every department in the organization has better data than you”you have more data in your Fitbit than you have about your learners!” you have more data in your Fitbit than you have about your pupils!

Perhaps learning professionals shouldn’t necessarily beat themselves up too much, though. Consider the size of the assignment that they face. In reality, learning is a hard thing to experience, let alone quantify. For example, it’s much harder to calculate than marketing (there’s no ‘buy’ button on a learning program). It also takes time and takes place against the backdrop of the fast-changing, frequently unpredictable market world of today, where no one has time to spare. When the performance of a person increases, it is often difficult to disentangle other variables such as incentivization or things occurring in their personal life from the role learning played in that progress.

Learning analytics is difficult for these reasons. It’s an entire field of academic research on its own, and one of fairly recent age, just over a decade old. Moreover, over the past sixty years, the training world has accumulated numerous refinements and revisions to its common four-step model created by Kirkpatrick and Katzell for learning assessment. And there is plenty to remember.

5 Top Big Data Pitfalls

Of course, to start making practical use of learning data, you don’t need to know all of it any more than you need a degree in mechanical engineering to drive a vehicle. But when you’re on the road to greater data literacy, there are several notable pitfalls to avoid; bear-traps that sometimes claim the unwary traveller. Here are five and how they can be stopped.

1. Failure to Manage Your Data

If you wait for the organization to ask for better details, there is a hypothesis called the ‘conspiracy of convenience’ that indicates that it will never happen. Analyst David Wilson and learning guru Charles Jennings advanced this idea, and it maintains that learning departments and their internal clients collaborate to avoid the topic of learning effects in a largely unspoken way. To assess if a learning intervention has any effect on results, the client does not ask for data and the learning department does not volunteer to provide it.

Nevertheless, if you spot this happening in your business and you go along with it, you will run the risk of irrelevance and finally obsolescence. Be proactive, then. Start making data a core part of your practice today and inform your internal and external clients about the advantages of a data-led approach.

2. Managing Too Many Factors

By taking a very narrow view of what can be achieved with learning data and thinking of it exclusively in terms of historical comparison of previous training programs, people sometimes lock themselves in.

We assume you should use a 360 approach to learning data to prevent this pitfall. Using just your rear-view mirror, you wouldn’t drive, so don’t limit your attention to learning assessment (important as it is): a more systematic approach would also include side mirrors, dash-cam, and a better view of the road ahead. Not all is about assessment or analytics.

So here’s a better box for your thinking: Donald Clark’s useful data schema in his book “Artificial Intelligence for Learning” will help you understand the full scope and potentially provided by addressing the complexities of Big Data. The scheme is based on objectives because, as Clark writes, “It is almost pointless to gather data if it remains unloved and unused. There must be a purpose to the endeavor.”

• Goal
• Example
• Describe
• Tracking and visualization of data for learning
• Analyze
• Evaluation, relation to business success and development of ROI in school, deciding how individuals learn best
• Foresee
• Quality estimation and whether individuals are likely to drop out
• Prescribe
• Engines of suggestion, adaptive learning

3. Doing the Same Thing Every Time

Many learners concentrate on unique data sources that are close-to-hand and reasonably easy to obtain, e.g. Your Learning Management System’s course completions and test scores, and neglect to look further afield. There is an ever-increasing volume of data from all kinds of sources available now. The disadvantage of sticking to restricted sources is that a clear causal relationship between one learning data source and an increase in performance is uncommon. In other words, when it comes to assessing effects, it is hard to find a ‘smoking gun’.

Solution? Solution? To analyze and take a ‘portfolio of proof strategy’ using various data sources. Chances are that you can only prove that your action could be the reason why anything has changed, but with more data points, you can stack that deck in your favour.

4. Lack of Collaboration

To ‘keep inside your path’ and overlook what is going on in the rest of the company may be all too tempting. Learning doesn’t happen in a vacuum, though. Learners are mostly day-to-day workers who take up far more time than their learning tasks, so it follows that the vast majority of information on their results, skills gaps, and expertise needs is not in the Learning Management System, but other areas of the company.

Organizations today are full of Big Data. Starting from where you are, use the data tools that are on hand in other parts of the market, but it may mean forming new partnerships and developing new skills to get hold of it.

5. Do-or-Die Thinking

It is all too easy for learning professionals to feel undervalued and overlooked, given all the requirements and stresses of dealing with the business and to start adopting a bunker mindset. In a scenario where the learning role has to open up and develop new skills to overcome the Big Data problems, the protective mentality this encourages is the very worst thing that can happen. Showing a gap in some kind of knowledge and skills will run the risk of being seen as a weakness. Yet you can’t be a competent learner and you don’t have the confidence to read!

Conclusion

Adopt an agile approach to fight the bunker mentality. Begin small, fail quickly, learn by doing. To support you, the Learning Pool has useful resources available. Download the eBook Data And Learning: Adding Learning Analytics To The Company if you want to learn more about the field of learning analytics. It covers all the basics, from selecting an appraisal methodology to using an LRS to reach performance targets. Join the webinar for more insight into learning analytics, and discover the real strength they hold.