Improving integrity in cyber education with interactive oral assessments

Along with 

The rise of artificial intelligence (AI) has transformed the landscape of education, challenging traditional assessment methods. As AI tools become more powerful and integrated into our everyday platforms, it’s clear that the old models of assessment can no longer be preserved. It’s time to embrace change and plan for the future that is already here.

Recognising the need for change

In an opinion piece((Justus, Z; Janos, Nik, D. (2024) Assessment of Student Learning Is Broken, Inside Higher Ed. Available at: https://www.insidehighered.com/opinion/views/2024/03/28/assessment-student-learning-broken-opinion (Accessed: 6 April, 2024).)) , Zach Justus and Nik Jano have made a compelling suggestion: the data we are currently collecting through traditional assessments are now worthless. Whether evaluating writing, STEM outputs, coding, or anything else, we are looking at a combination of student and AI work. This realization necessitates a wholesale re-evaluation of how we measure student learning.

The call for leadership and a pause

To address this challenge, Justus and Jano propose two major changes. First, accreditation bodies, both regional and discipline-specific, need to show leadership in adapting to this new reality. Second, they suggest a one-year pause wherein no assessment is expected or received. Instead, educators should focus on meeting, thinking, and working on what assessment of student learning looks like in this new world.

Embracing change with hope, not fear

As Director of Notion Digital Forensics, Matt O’Kane emphasises the importance of responding to this change with hope, not fear. There are solutions, and we can learn from the experience of running live high-stakes assessments with Masters level cyber students with leading researcher Dr. Pranit Anand.


Interactive Oral Assessments: a promising solution

One promising approach is the use of interactive oral assessments. Dr. Pranit Anand will be facilitating a session on this topic on Monday, April 8th. The session will draw on the work of A/Prof Popi Sotiriadou and the experiences of Dr. Anand, Matt O’Kane, Emina Abroon, Amita Krautloher, and Marcelle Droulers in developing and implementing interactive oral assessments.

Notion Digital Forensics’ Contribution

Dr Anand has already commented on how well this worked, given positive comments from students. This may suggest that this approach may be a viable way forward in the age of AI.

Embracing the Future of Education

As AI continues to shape the educational landscape, it’s crucial that we adapt our assessment methods to reflect this new reality. By embracing change, showing leadership, and exploring innovative approaches like interactive oral assessments, we can ensure that our educational systems remain relevant and effective in the face of technological advancement.

You can get some ideas on how to run this from a guide jointly developed by Dr Anand, fellow researchers Emina Abroon (UNSW Business School), Amita Krautloher, and Marcelle Droulers of Charles Sturt University and myself.

Oral Assessment | UNSW Teaching Staff Gatewayhttps://www.teaching.unsw.edu.au/assessment-methods/oral-assessment

Important note on general advice

I am a cyber security specialist, but I may not be YOUR cyber security specialist. 

All cyber-security and digital forensics decisions require careful consideration of your own circumstances and risks. General information is not not tailored to your individual needs. You should seek the advice of a suitably qualified cyber-security or digital forensics specialist.