The Remote Proctored Exams Dilemma
Around the world, the advent of virtual environments and remote working has opened up new horizons of possibilities within educational settings.
One increasingly being explored by institutions and educators is that of remote proctored exams.
However, while offering real potential, such exams also present us with a dilemma – not just whether they should be used, but also how they should be used. In this article, we will look at what remote proctoring is, its advantages and disadvantages, and how the views we hold of remote proctoring can be shaped by our perspectives. Finally, we show how taking an assessment security perspective can offer ways to get the most out of remote proctored exams.
Remote proctoring – what is it, and what do people think about it?
So, what is remote proctoring? For the purposes of this article, we will define it as follows:
- A timed exam,
- conducted on a student-provided computer,
- at a location of the student’s choosing, that is
- monitored or recorded by a person and/or computer.
Elements of biometric or identity verification will often be present, and a third-party provider is usually involved as well.
From there, things become more complicated - attitudes towards remote proctoring vary but according to Philip Dawson, often fall into one of the following five categories:
- Students do worse (so remote proctoring is better): Probably the most common type of remote proctored exam paper claims that “students do worse when they're proctored”, and then proceeds on the assumption that this is because they are cheating less.
- Students do the same (so remote proctoring is okay): In contrast, other papers argue that many students don’t do worse when proctored, and so that means it's okay.
- Authentic assessment would be better: Others start from the position that as people cannot cheat in authentic assessment, it would be preferable to “abandon exams and do authentic assessment instead”. Unfortunately, Cath Ellis and other researchers have shown that you can indeed cheat in authentic assessment.
- “Policing” technologies are antithetical to education: There’s also the type of paper that strikes an ‘anti-surveillance, anti-application of policing-type technologies’ stance to education.
- Trust our students: Finally, there are those studies that argue that trust is the answer; that if we just place enough trust in our students, they won’t cheat.
Given such divergent views, it is unsurprising that there is little consensus on the advantages of remote proctoring, and whether or not these sufficiently outweigh the disadvantages.
Remote Proctoring: Advantages
So, what are the advantages of remote proctoring? In the opinion of some, the advantages are just one: that remote proctored exams can detect or deter cheating in online exams. According to this view, there isn’t really another, because everything else you might come up with – for instance, that students prefer to type rather than write, or that they prefer to do it at home or that there's rich media affordances if we do online exams – all that is about online exams rather than specifically about remote proctoring.
What if we focus on the effectiveness of remote proctoring in detecting and deterring cheating? What evidence might there be to support this? Many studies argue that because students finish more quickly in remote proctored exams and have poorer outcomes, it perhaps suggests that they’re being prevented from cheating. But there are other potential explanations. People may be more anxious in remote proctored exam, for instance, and this causes them to finish more quickly and make more errors. Taken on its own then, this isn’t really a convincing argument that remote proctored exams reduce cheating. What's also missing is peer-reviewed evidence proving that proctoring detects cheating. An important source active in this area is security researcher Reilly Chase, who ran remote proctored exams inside a virtual machine and succeeded in highlighting their vulnerability.
In summary then, although some evidence exists that suggests remote proctoring can deter cheating, many studies simply don't support the inference. At best, we can say that remote proctoring may be a good thing, that it may help deter cheating, and it may help detect cheating. To settle the matter definitively, however, more and better research will be needed.
Remote Proctoring – Disadvantages
What, then, about the disadvantages of remote proctoring? Unlike advantages, these are many and varied. In the literature, a range of studies have looked at the idea of proctoring as surveillance or as an invasion of privacy. Besides the immediate sense of intrusion people may feel, there are other questions, such as those over GDPR, the use of personal information, and the transfer of data across borders and between different data control regimes. Others revolve around the use of data forensics - I may consent to my data being stored on a system, but am I happy to permit the running of certain algorithms on that data?
Some oppose proctoring as a kind of surveillance (albeit one actively participated in by people) while others argue against it on the grounds of higher stress, discomfort and anxiety for students. Others again view remote proctoring and similar technologies as destructive of trusting relationships between educators and students, or as being in some way racist, ableist or discriminatory.
Finally, another major critique of remote proctored exams is that they allow us to continue doing what we've always done, and that this may not be a good thing. If we really want to reimagine our practices around learning and assessment, are remote proctored exams an obstacle to our doing this?
To date, however, it has proven difficult to establish these disadvantages empirically, and it’s possible that some of these may arise more out of the way we conduct remote proctored exams, rather than being inherent features of this approach.
Differing Perspectives
As the above suggests, comparing the advantages and disadvantages of remote proctoring is not straightforward. A better approach may be to bring a greater awareness to the power of perspective, and the ways this can shape our differing responses to this subject.
How different can these perspectives be? According to Dawson, they “run the gamut from the criminological to the critically pedagogical. From those who think that proctoring can reduce cheating through the threat of penalties, to those who believe that proctoring does such harm that it’s fundamentally incompatible.” There is a spectrum between these two extremes. For those taking more of a cybersecurity approach, the focus might be on gathering the hard evidence necessary to prove proctoring detects cheating. Others prioritizing academic integrity would perhaps ask whether remote proctoring fails to give students the opportunity to develop as individuals with integrity. Then there are those who adopt what might be called a “socio-technical view”, where we need to understand the everyday realities of proctoring for students and staff. Finally, there's what might be called a surveillance studies view: that proctoring is problematic in how it contributes to a wider surveillance culture that adapts students to a world where being surveilled is okay.
Where do we begin?
One approach, advocated by Philip Dawson and developed as a resource for TEQSA, is centered around assessment security but goes beyond this to include academic integrity as a key aspect. It involves measures actively developed to harden assessment against cheating attempts, includes approaches to detect and evidence such attempts, and incorporates additional measures to make cheating more difficult.
It’s important to note that, in the assessment security view, any single approach to address cheating will be ineffective. Instead, a constellation of approaches is essential, consisting of lots of different layers. In this approach, the focus is on striking a balance between trust on the academic integrity side, and detection in assessment security. In practice, this means that we want to educate students on how to care and do the right and ethical thing. We complement this with a proactive view on assessment security, to ensure people are not credited for things they haven't done. This is where remote proctoring comes into the picture, but within a very clear framework:
-
Remote proctored exams are a last resort: We never jump straight to remote proctored exams. If there's some other task that we could use instead that will have good enough assessment security, it's going to suit our purpose better.
-
Exam designs are sound assessments of learning: We never want a remote proctor assessment for learning. But when we're doing assessment of learning, where we’re asking if the person hit the learning outcomes and to what extent then, remote proctoring might be valid.
-
Only the minimal restrictions required are used: We want to see if somebody is really doing this thing in the world, what restrictions do they do it under? Our focus should be on authentic assessment.
-
Students are offered an alternative: they could be offered to sit the remote proctored exam at home or in an exam hall.
-
Equity, diversity, adversity and accessibility are catered for: students might have insecure housing, or a disability, or might be going through a really difficult point of time in their life. All those things need to be considered.
-
Providers pilot remote proctored exams adequately before using them in assessment, to avoid avoidable mishaps.
-
A whole-of-institution approach is taken: a single lecturer can't just decide to use remote proctored exams, as there's more than any one person could do. We need a wider spectrum of expertise: IT teams, learning and teaching staff, legal teams, and learning designers.
-
Regulatory requirements and standards around privacy and data security are met.
-
Effective governance, monitoring, QA, evaluation and complaints procedures are in place.
-
Staff and student capacity building and support are available and ongoing: this isn't a skill set that everyone's born with to use these things. We need the support to be there. We need it to be ongoing.
As with all technologies, remote proctoring presents us with positives and negatives, advantages and disadvantages, opportunities and challenges.
The purpose of this article has not been to advocate for one or other side of the argument. Rather, it has been to call to mind this topic’s multifaceted nature, and to encourage us to remain open to other ideas, approaches and perspectives.
In this way, we can take steps to ensure this technology serves the needs and interests of teachers, learners, and the educational system as a whole, rather than vice versa.
