Two major challenges are hitting legal education at the same time: the NextGen Bar Exam and AI. Both feel threatening. This post offers an alternative to the natural human reactions to threats: fight, flight, or freeze. Instead, I propose combining the two, AI and a NextGen Bar testing format, to create a teaching tool for first-year law students.
One NextGen testing format will involve integrated question sets that include multiple questions revolving around an evolving legal problem. See this link for examples. Those examples include, among other things, tasks that engage the students in identifying mistakes in a legal document. For example, Example Question Set 2 on the website of the National Conference of Bar Examiners includes a complaint and the following task:
[F]ind five mistakes in the complaint. For each mistake, (1) identify the paragraph that includes the mistake, (2) describe the mistake, and (3) explain how you would correct the mistake.
A mistake can be a mistake of fact, a mistake of substantive law, or a violation of the provided procedural rule. Missing information is not a mistake. Assume that the following are correct: (1) all bracketed material and (2) all formatting and numbering.
The advent of AI requires lawyers to engage in just such tasks, i.e., to identify and correct mistakes made by AI in generating a draft work product.
My idea is to provide students, on a regular basis, AI-generated analyses of the shorter hypotheticals I regularly ask in class.
For example, early on in my contracts class, I ask students to argue whether the language in a series of problems expresses the requisite commitment to constitute an offer. I use the same terms for the hypotheticals so that students focus on the language, rather than on other factors. Students struggle to explain why “I will . . .” almost always expresses the requisite commitment, why “I cannot . . .” does not, and why more ambiguous language like “I ought to . . .” does not, upon analysis, express the requisite commitment.
This fall, I will, from time to time, provide students with answers to my hypotheticals and ask them to identify and explain a specified number of errors. I will ask my students if these exercises help them understand the reasoning process better; in the past, I have observed that at least some students can see errors in others’ work that they cannot see in their own work.
This find-three-errors method could also be used as a Classroom Assessment Technique, a wonderful teaching practice I explained in this past teaching tidbit. In other words, at the end of a class session in which you focused on a specific skill, you distribute a short work product that involves application of the skill but includes one or two flaws. You can ask your students to draw a circle and explain the problem with each flaw. The results will give you useful information about whether your students made progress in developing the skill.
Interesting experiment. As an experiment, I asked the Claude AI to generate a flawed answer to the “I ought to . . .” hypothetical. The first one Claude created introduced other issues (beyond offer) so I asked it to try again. (My students often make the mistake of blending issues together.) With coaching, Claude was able to generate the flawed version I will use in class. For fun, I asked Claude to generate a model analysis. Claude’s first try made a mistake my students often make: Claude said that the words “I ought to . . .” “refer to something that the speaker believes they ought to do.” Defining a word with itself is perfectly circular and (obviously) is a flawed analysis. When I pointed that error out to Claude, it agreed that it had made an error and generated a solid answer. I am not sharing Claude’s work here as I am confident my students will have the googling skills to find this blog post. However, if you email me, I will send you Claude’s work.
One Last Point. This week’s teaching tidbit was conceived during a conversation I had last week with a thoughtful and inspiring colleague, Professor Yvette Pappoe of the University of the District of Columbia David A. Clarke School of Law. We started by repurposing the idea in last week’s blog post, cognitive think alouds, into a plan for Professor Pappoe to have her students trace their thinking via cognitive think alouds with her TAs as they analyze a hypothetical. Professor Pappoe hopes that, when the student speak aloud their thinking as they start analyzing a hypothetical, they will reveal missteps in their thinking for which her TA’s can provide coaching. Somehow, after discussing that idea, we ended up discussing the idea addressed in this post. My takeaway is that discussions of teaching, even when we are exploring someone else’s idea, almost always make us at least a little better.
