Josh Blackman (South Texas), How To Assess AI-Aided Students?

My tentative plan for ConLaw in the spring is to switch to an oral, in-person midterm.

Taking a step back, I have long struggled with how to use midterms effectively. The most useful aspect of a midterm is to provide an early intervention–feedback for students who need help. But by the time the midterm is graded, it is often too late in the semester to correct course. (I do not give multiple choice questions in ConLaw, though that format would make it far easier to grade quickly.) When I started teaching, I would proctor the midterm right before spring break, and then spend the entire spring break grading midterms, and return them when we got back. Needless to say, with family commitments, that is no longer an option. The other problem with midterms is that some students have not yet hit their stride by the seventh week. Indeed, there was not always a clear correlation between grades on the midterm and grades on the final.We need to reconsider oral evaluations.

All of these concerns remain. Still, in light of the threat to assessments posed by AI, I will need to redouble my efforts. Here is my working approach:

[H]alfway through the semester, I will proctor an individualized, oral midterm in my office. Each session will last approximately five minutes. If I do ten sessions per hour, with breaks, I can wrap up the entire process in a single day. There is an advantage to doing everything in a short period of time to reduce the risk of students talking to each other. At my law school, 1Ls do not have classes on Friday, so that timing would work well. In the first instance, I would randomly assign students to slots throughout the day. If they have an actual, documented conflict, they can reschedule. And there will be two weeks of advance notice, which should help reduced potential conflicts.

I will need to give some more thought about how to handle timing for accommodated students. As most professors know, the number of students who have testing accommodations has skyrocketed in recent years. Often accommodations are provided for students who are not able to write with a pen or type with a keyboard–so-called “scribing.” But the oral exam eliminates that need. Students would only need to listen and speak. …

I would also record and digitally transcribe the entire session, should any disputes arise later. The midterm would not be worth a large portion of the grade. I think all law schools permit professors to vary grades a small amount based on participation. I would make that participation score turn on the performance of the in-person midterm. (I’ve never figured out how to accurately and consistently record class participation over the course of a semester, so I don’t try.) …

Using an oral midterm is sort of an interim solution. The ultimate move would be to make the final exam an in-person oral exam. A challenge is anonymity. While final exams require anonymity, it is less important for midterms. Professors are expected to go over a student’s midterm to provide one-on-one feedback. But perhaps as so much of the work students submit is no longer their own, but instead created by AI, eliminating the cloak of anonymity may be the final way to determine what a person actually knows.

Prior TaxProf Blog coverage:

• UT Law on AI
• Legal Education Existentialism, No. 1
• Legal Education Existentialism, No. 2
• Legal Education Existentialism, No. 3

Editor’s Note:  If you would like to receive a daily email with links to legal education posts on TaxProf Blog, email me here.