Skip to main content

Building GenAI into Law Assessments: How do students use genAI when given the chance?

Dr Samantha Kontra, Senior Lecturer, Flinders University

Abstract​

Generative artificial intelligence (‘genAI’), such as ChatGPT,1 has taken the world by storm. It has also placed added stress on academics in higher education as we grapple with creating genAI-proof, authentic assessment items. While some legal academics have embraced genAI, others have tried to ignore it,2 and some have tested its boundaries leading to conclusions that ChatGPT can produce passable answers in some law topics,3 and can draft legal documents, though it is challenged by legal reasoning.4 Students, too, use genAI for a multitude of purposes, both internal and external to their law studies, sometimes leading to excellent learning opportunities.5

This presentation explores the incorporation of genAI in classroom teaching and assessment in an undergraduate law program in 2023. It identifies the ways in which first year students in a foundational law topic purport to use ChatGPT, and reports on the ways in which they actually use ChatGPT during in-class formative exercises as well as in summative assessments. This is contrasted with penultimate and final year students’ use of genAI (ranging from ChatGPT to Amica6) in two authentic practice-based assessment tasks in a Family Law elective topic, as reported during assessed supervisor interviews. Ultimately, this research has found that first year students are more likely to use genAI when permitted, but with a specific focus on planning and initial idea generation. Senior students are much less likely to use genAI and have poor opinions of genAI use in a legal context. The analysis of these responses has led to the development of a specific module in a first year foundational law topic that introduces students to genAI and its uses and implications during law school and in the legal profession.7

This presentation will include opportunities for interactive discussion and sharing of ideas about law student use of genAI and creation of law assessment items in the genAI era.

Footnotes​

  1. OpenAI, Introducing ChatGPT (Blog, 2015-2024) <https://openai.com/blog/chatgpt>.
  2. Carolyn V Williams, ‘Bracing for Impact: Revising Legal Writing Assessments Ahead of the Collision of Generative AI and the NextGen Bar Exam’ (2024) 28 The Journal of the Legal Writing Institute 1.
  3. Stuart Hargreaves, ‘Words are Flowing Out Like Endless Rain into a Paper Cup’: ChatGPT and Law School Assessments’ (2023) 33(1) Legal Education Review 69.
  4. Williams (n 2).
  5. Hargreaves (n 3).
  6. National Legal Aid, ‘Helping You to Separate Smarter’ Amica (Web page) <https://amica.gov.au/>.
  7. Williams (n 2); Paul W Grimm, Maura R Grossman and Cary Coglianese, ‘AI in the Courts: How Worried Should We Be?’ (2024) 107(3) Judicature 65.