It’s difficult to be precise as to how much data we collectively generate. In 2013, an IBM report estimated that we are creating 2.5 quintillion bytes of data every day. To put that in context, that 1GB Flash drive you keep in your desk drawer – we’re filling 1 billion of those every day. With the increasing use of cloud services, the Internet of Things and social media worldwide, it is safe to assume that this figure continues to grow exponentially.
So how can the 21st century obsession with data help us in the world of educational assessment?
To answer this question, let’s go back to basics and ask “what is education and assessment actually for?”:
- To give our children and young people a well-rounded, broad-ranging view of the world?
- To give our children and young people the essential skills they’ll need to succeed in adulthood?
- To make our future workforce innovative and efficient for the sake of the economy?
All of these are valid justifications for educating and assessing, but how do we know that we are educating young people effectively? And how can we tell when our approach to teaching needs to change or when intervention is necessary?
This is where data analysis can help. But before we look at why data is a good idea, we need to understand where the idea of data itself comes from.
Why do we need data?
Epistemology is the study of knowledge itself and focuses on three areas: rationalism, scepticism, and empiricism. Here’s what they are and how each perspective could be interpreted in the world of educational assessment:
- Rationalism (all about deductive reasoning)
Good teachers have taught some stuff and therefore the pupils have learnt it. If the pupils have learnt, then they know what they need to know. We won’t bother to assess them.
- Scepticism (all about the fact that knowledge is philosophically impossible)
There’s no way to be certain whether teachers have taught anything or whether pupils have learnt anything. In fact, anything that pupils may or may not have learnt is categorically untrue and can’t be proven anyway. What’s the point in teaching or assessing anything?
- Empiricism (all about evidence)
We’ve set some criteria and then measured what actually happened. Teachers taught pupils how to count to 100 and pupils have demonstrated that they can count to 100. We can test against this measure and examine the outcome.
Modern science is underpinned by the ‘scientific method’, which is based on empiricism. This approach of stating a hypothesis or assumption, testing it and adapting the approach based on the outcome, has been used for the last 300 years or so. It comes as no surprise therefore that most awarding bodies and governmental institutes follow the same pattern for defining policy or curriculum frameworks.
Measuring and evaluating data
Now we understand why we need data, the next step is deciding what to measure and evaluating what meaning there is in the data that we collect. One of the biggest challenges in this regard is a phenomenon known as ‘deductive fallacy’. Essentially, each bit of data is correct but the conclusion that we derive from it is false, because our deductive reasoning is flawed. For example:
I like eating carrots.
Rabbits eat carrots.
Therefore I am a rabbit.
I can drive a car.
Rabbits can drive cars.
Before long, we run the risk of drawing a multitude of false conclusions from a series of false deductions.
Finding meaning in data is what keeps our data analytics team busy working on the Performance Tables each year with the Department for Education. Analysing some of the rich data captured in RM Assessor around marking statistics also gives us and our customers numerous possibilities to monitor and improve quality assurance and consistency in marking. In each case, avoiding misleading or illogical conclusions is always a principle consideration.
Ensuring accurate data in the future
In a future that sees us generating more and more data, and a world that relies heavily on algorithmic analysis and determination, the debate is no longer about whether or not we should use data in education. The debate is about how we ensure that the types of data, the insights that we derive, and the extent to which we rely on machines to answer questions that we didn’t even know to ask, is accurate and relevant.
With the continuing evolution of Artificial Intelligence and Machine Learning and its increasing presence in the world of education, the conversation will become centred on the efficacy of the policies, interventions, assessment outcomes and statistical analyses. Collecting the data is now a given.
As educators, technologists, parents and human beings, our challenge is to ensure that we imbue the AI that we are creating with enough wisdom and judgement that, with each passing 2.5 Exabytes of data per day, we don’t end up giving rabbits driving licences.
Getting it right, however, will reveal a wealth of insights into the effectiveness of education and assessment, the likes of which has never has been seen before. The opportunity to positively impact the education experience of our young people is huge and we should embrace it.