In December I introduced a framework to support faculty with the complexities of incorporating generative AI into their assessments. One feature of this framework encourages the consideration of separating assessment components into AI-active and AI-inactive sections to better articulate expectations of this technology use for students. Boud et al.’s (2018) conception of assessment for learning broadened my perspective on how the Artificial Intelligence-Supported Assessment (AI-SA) framework may be applied to a variety of tasks faculty use to gauge students’ progress toward learning goals.

In a prior blog post, I shared framework rankings of an AI-SA called Using ChatGPT to Explore Calculus Applications and considered the alignment of these rankings with student feedback on a short survey about their learning experience. Survey results provided some evidence of students’ achievement of a calculus learning goal and their productive use of generative AI. In this post, I will share the rest of my analysis of student work on Using ChatGPT to Explore Calculus Applications.

When designing student expectations of that AI-SA, I attempted to capitalize on the capability of generative AI to produce multiple examples of a concept or application. I asked students to first pick an area they were interested in such as a hobby, science, or sport, and then use ChatGPT to produce examples of how the calculus concepts they were investigating applied to their area of interest. Students were asked to state their major in the writing portion of this AI-SA. The table below contains a summary of students’ majors and the areas of interest they chose.

Summary of Students’ Majors and Areas of Interest

Nineteen different areas of interest emerged with the only repeats being music and marketing. Students chose a variety of areas to investigate calculus applications, about half of which aligned with their major and the others to another context. I was encouraged to see that student choice built into this AI-SA resulted in this diverse set of interest areas and many different applications of calculus. In previous semesters I typically applied this calculus content to only three to five different areas during class sessions or in assessments.

Seeing students’ individual interests honored when they used ChatGPT reminded me of some characteristics of culturally responsive teaching (CRT). Wlodkowski and Ginsberg (2018) describe CRT using a four-component framework that includes enhancing meaning with engaging and challenging learning experiences that incorporate learners’ perspectives and values. Students’ interests being centered in this assessment also reminded me of Boud et al.’s (2018) view of assessment as practice. In this view, faculty should think about assessment in ways that make visible many students’ needs and concerns such as how disciplinary content is relevant to their lives.

Details included in student writing submissions on this AI-SA gave evidence to how they used ChatGPT to make calculus content responsive to their perspectives and values while also evidencing achievement of the specific learning goal. One student decided to investigate music in relation to the calculus learning goal of establishing the connection between rates of change and accumulation. In the table below, I include excerpts from his writing product along with samples of feedback I provided.

Summary of Student’s Writing Samples and Instructor’s Feedback

Samples from this student’s writing submission confirm many of the quality rankings shared for this AI-SA in the previous blog post. When giving feedback I thought about Clark & Talbert’s feedback loop; the calculus learning objective; and my initial framework rankings.

(Clark & Talbert, 2023, p. 12).

This feedback loop and my specific learning intentions helped me to focus my feedback. I paid attention to aspects of students’ writing and thinking that went beyond just identifying interesting points or well-worded paragraphs to account for the calculus learning goal and student use of ChatGPT. Based on this student’s performance on subsequent assessments it appears that he used this AI-SA and the feedback I gave him to adequately prepare himself to demonstrate understanding through the remainder of that semester.

In closing, consider some takeaways I have been appreciating during this work. First, faculty who want their students to use generative AI in meaningful ways need to be purposeful and detailed about the written expectations they include in their assessments. After analyzing student writing submissions on this AI-SA, I had the distinct impression that students used ChatGPT in ways I desired due to the detailed guidelines found in the assignment itself. I further had the impression that students are ready to step up to the challenges of using generative AI if they are provided appropriate guidance (e.g. ChatGPT-Active and ChatGPT-Inactive components) and scaffolds in their efforts. Finally, the work of creating assessments for learning that incorporate student use of generative AI is no small feat. I suggest that interested faculty start with one assessment and adapt it for student generative AI utilization. Deeply consider student work and feedback, then revise and try it with other students. Email me at atrocki@elon.edu and let me know how it goes.

References

Boud, David, Phillip Dawson, Margaret Bearman, Sue Bennett, Gordon Joughin, and Elizabeth Molloy, 2018. “Reframing Assessment Research: Through a Practice Perspective.” Studies in Higher Education 43(7): 1107-1118.

Clark, David & Robert Talbert. 2023. Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education. New York: Stylus Publishing, LLC.

Wlodkowski, Raymond J., and Margery B. Ginsberg. 2018. Enhancing Adult Motivation to Learn: A Comprehensive Guide for Teaching All Adults (4th ed). Jossey Bass.

Aaron Trocki is an Associate Professor of Mathematics at Elon University. He is the CEL Scholar for 2023-2024 and is focusing on models of assessment and feedback outside of traditional grading assumptions and approaches.

How to Cite this Post

Trocki, Aaron. 2024. “Utilizing a Framework for Artificial Intelligence-Supported Assessments: Part 2.” Center for Engaged Learning (blog), Elon University. March 12, 2024. https://www.centerforengagedlearning.org/utilizing-a-framework-for-artificial-intelligence-supported-assessments-part-2.