A team within RM Results was tasked with the project to understand what opportunities there were for the use of Artificial Intelligence (AI) in e-marking. This blog post gives an overview of their research findings.

Artificial intelligence has long been seen as a panacea technology that will usher in a new era for human development. In reality, it’s 2018 and AI is already disrupting established industries – you only need to look at Ford’s claim that it will have driverless cars on the road by 2021[1] for a powerful example of disruption due to innovation in AI. A recent report from PwC states “UK GDP will be up to 10.3% higher in 2030 as a result of AI – the equivalent of an additional £232bn”[2].

What is AI?

Simply put, the term artificial intelligence refers to software systems that are capable of performing tasks traditionally thought to require human-levels of intelligence. That said, numerous definitions of the term exist. It is a broad subject, areas of active research include:

  • Natural Language Processing – systems that can process written language and make deductions from it.
  • Machine Learning – algorithms that are capable of learning how to perform a task, based upon being given lots of examples.
  • Deep Learning – similar to Machine Learning but with algorithms that are much less task-specific and more powerful, Deep Learning systems are generally built to mimic the way our brains process information
  • Computer Vision – systems that can make deductions based upon video and images.

One of the keys to unlocking the massive opportunity in AI is having rich sets of data. AI algorithms can identify subtle trends in data to offer valuable insights and future predictions.

It is evident that AI offers massive opportunity to redefine conventional work processes, and assessment will be no different. So, what should we expect of AI in assessment? We’ve pick out two applications that we find particularly interesting:

1. Augmented marking

Marking a long-answer or essay style exam script can be a difficult and time consuming procedure: ensuring you allocate marks accurately and consistently between scripts whilst balancing the fact that you have another ten to mark before bedtime! The good news is, AI is here to help.

Recent advances in AI applications – Natural Language Processing and Video Analysis for example – could soon be used to help guide a marker with timely and consistent mark allocation, for example:

  • Picking out the sentiment of a text – did the candidate write in the correct tense?
  • Phrase analysis – did the candidate use a similar phrase to candidates that have previously been allocated a certain mark?
  • Facial blurring – Focusing only on the correct student in a digital media assessment
  • Keyword analysis – has the candidate used the correct variation of keywords?
2. Aggregated marking

Short answer assessments could also be aided by advances in AI. Examiners often have to mark thousands of responses to a certain question, where the correct answer can only be a small variation of a word or phrase. Using AI technology, it could be possible to group similar responses that could all receive a given mark. This would reduce the marking burden from thousands of responses, to hundreds or possibly even tens of responses. Candidate feedback could be much quicker and the consistency of marking would be significantly improved.

A five level model for the adoption of machine marking

Like many, we at RM Results can see the future potential for AI in resolving many issues faced by those working within the assessment industry, and this project sought to better understand the opportunity for AI in e-marking specifically. One of the project outputs was a model that maps the likely phases of adoption of AI in the sector. It starts with improving backend processes and moves on to augmenting the human marking experience, finally progressing to the stages of automated marking.

To find out more about the five levels of adoption to fully automated machine learning, as well as what possibilities there are for AI in e-marking, download your free copy of ‘The opportunities for artificial intelligence in e-marking now and in the future‘.

[1] https://media.ford.com/content/fordmedia/fna/us/en/news/2016/08/16/ford-targets-fully-autonomous-vehicle-for-ride-sharing-in-2021.html
[2] https://www.pwc.co.uk/economic-services/assets/ai-uk-report-v2.pdf