Using Generative AI for Assessment Design, Evaluation and Feedback: Opportunities and Challenges
Cornelia Koch, University of Adelaide
Abstract
Generative AI technology is disrupting the world as we know it. A myriad of tools is developing with incredible speed that will revolutionise service industry sectors, including education and law. The advent of large language models (e.g. ChatGPT) sent shock waves through the University sector in 2023. Of immediate concern for legal academics was academic integrity of assessments: would generative AI tools undermine our ability to use ‘traditional’ types of assessment such as essay style or problem based written assignments? Would ChatGPT or its ‘cousins’ write assignments for our students, meaning that they would not learn relevant professional skills? Most universities scrambled to develop policies for students on working with AI responsibly. Most course coordinators changed their assessment schemes to ‘AI proof’ the integrity of assessment. However, after this initial ‘shock’, the potential for generative AI to contribute positively to learning and teaching must be considered.
While we must ensure responsible and ethical use of generative AI tools and prevent breaches of academic integrity, we should also recognise the countless opportunities that generative AI tools create as assistants in the learning and teaching space. Tools can assist academics with many aspects of their work, for example, course design, content creation, teaching delivery, assessment design, evaluation and feedback. This paper focusses on using generative AI tools (ChatGPT4, Microsoft Copilot and others) in the assessment space. It reports on a case study where I have used these tools for drafting multi-answer quiz questions and feedback. It also explores ways for developing scenario-based assessment, providing formative feedback to students during completion and assisting markers with providing summative feedback. While celebrating these opportunities, the paper also reflects on challenges of using generative AI tools, including data privacy, data sovereignty and intellectual property issues. Overall, the paper argues that harnessing the power of generative AI for learning and teaching is very valuable for academics and students, as long as tools are used ethically and appropriately.
This requires clear policies and a commitment to upskilling university staff and students in the use of generative AI.