HomeBlogFeedback Utilizing a Framework for Artificial Intelligence-Supported Assessments: Part 2by Aaron TrockiMarch 12, 2024 Share: Section NavigationSkip section navigationIn this sectionBlog Home AI and Engaged Learning Assessment of Learning Capstone Experiences CEL News CEL Retrospectives CEL Reviews Collaborative Projects and Assignments Community-Based Learning Diversity, Inclusion, and Equity ePortfolio Feedback First-Year Experiences Global Learning Health Sciences High Impact Practices Immersive Learning Internships Learning Communities Mentoring Relationships Online Education Place-Based Learning Professional and Continuing Education Publishing SoTL Reflection Relationships Residential Learning Communities Service-Learning Student-Faculty Partnership Studying EL Supporting Neurodivergent and Physically Disabled Students Undergraduate Research Work-Integrated Learning Writing Transfer in and beyond the University Style Guide for Posts to the Center for Engaged Learning Blog In December I introduced a framework to support faculty with the complexities of incorporating generative AI into their assessments. One feature of this framework encourages the consideration of separating assessment components into AI-active and AI-inactive sections to better articulate expectations of this technology use for students. Boud et al.’s (2018) conception of assessment for learning broadened my perspective on how the Artificial Intelligence-Supported Assessment (AI-SA) framework may be applied to a variety of tasks faculty use to gauge students’ progress toward learning goals. In a prior blog post, I shared framework rankings of an AI-SA called Using ChatGPT to Explore Calculus Applications and considered the alignment of these rankings with student feedback on a short survey about their learning experience. Survey results provided some evidence of students’ achievement of a calculus learning goal and their productive use of generative AI. In this post, I will share the rest of my analysis of student work on Using ChatGPT to Explore Calculus Applications. When designing student expectations of that AI-SA, I attempted to capitalize on the capability of generative AI to produce multiple examples of a concept or application. I asked students to first pick an area they were interested in such as a hobby, science, or sport, and then use ChatGPT to produce examples of how the calculus concepts they were investigating applied to their area of interest. Students were asked to state their major in the writing portion of this AI-SA. The table below contains a summary of students’ majors and the areas of interest they chose. Summary of Students’ Majors and Areas of Interest StudentMajor/Intended MajorArea of Interest1Mechanical EngineeringEngineering2FinanceMarketing3Strategic CommunicationsMotion Pictures and Visual Effects4Strategic CommunicationsSoftball5Business AnalyticsSports Marketing6MarketingMarketing7International BusinessRunning a Half Marathon8EconomicsEconomics9Finance and AccountingPopulation Growth10AstronomyLuminosity of Celestial Bodies11Electrical EngineeringTelevision Show Viewership Ratings12MarketingMarketing13PhysicsParticle Motion14Psychology & AccountingAccounting15Finance and Business AdministrationFootball16Mechanical EngineeringTennis17Math with Secondary Teaching LicensePsychology18Exercise SciencePhotography19BiologyMusic20Cinema & Television ArtsMusic21EconomicsBusiness22FinanceAutomobiles Nineteen different areas of interest emerged with the only repeats being music and marketing. Students chose a variety of areas to investigate calculus applications, about half of which aligned with their major and the others to another context. I was encouraged to see that student choice built into this AI-SA resulted in this diverse set of interest areas and many different applications of calculus. In previous semesters I typically applied this calculus content to only three to five different areas during class sessions or in assessments. Seeing students’ individual interests honored when they used ChatGPT reminded me of some characteristics of culturally responsive teaching (CRT). Wlodkowski and Ginsberg (2018) describe CRT using a four-component framework that includes enhancing meaning with engaging and challenging learning experiences that incorporate learners’ perspectives and values. Students’ interests being centered in this assessment also reminded me of Boud et al.’s (2018) view of assessment as practice. In this view, faculty should think about assessment in ways that make visible many students’ needs and concerns such as how disciplinary content is relevant to their lives. Details included in student writing submissions on this AI-SA gave evidence to how they used ChatGPT to make calculus content responsive to their perspectives and values while also evidencing achievement of the specific learning goal. One student decided to investigate music in relation to the calculus learning goal of establishing the connection between rates of change and accumulation. In the table below, I include excerpts from his writing product along with samples of feedback I provided. Summary of Student’s Writing Samples and Instructor’s Feedback Excerpts from Student’s WritingSamples of Instructor FeedbackAn area of interest that calculus can be applied to is music. I chose this subject because music has been a big part of my life. I was in the boys’ choir from second to seventh grade, I played in the school band, I sang in the top choir for my high school, and I just love music of all sorts and all genres. Before applying calculus to music, we have to remember that two of the biggest concepts in calculus are rates of change and accumulation.Excellent job providing your area of interest and explaining how it relates to your background and life. You’ve also prepared your reader for the paper’s purpose to apply rates of change and accumulation from calculus to your area of interest.ChatGPT is also good at providing connections between calculus and anything you want. When asked, “How do rate of change and accumulation in calculus apply to music?” I was provided with eight different examples of either rate of change, accumulation, or both. Each example had a brief explanation to provide more information. These examples are shown below:You have used ChatGPT in an effective way to generate eight different examples of how these calculus concepts apply to music. I particularly enjoyed reading about digital signal processing, which is an application I had not explored before.From what ChatGPT put together for that question, it seems clear that rates of change are related to music by how fast or slow the tempo, wavelengths of a soundwave, dynamic, processing, pitch, and more change. This leads into accumulation being how the different rates of all of these components add to the “cumulative affect” (as ChatGPT puts it) of the sound of the music. These concepts can be applied because a greater rate of change of tempo, dynamics, etc. gives an effect of more energy to the overall sound (the accumulation), while a smaller rate of change of those things give an effect of less energy to the overall sound.You have explained a significant example of rates of change and their relationship to accumulation. I encourage you to think about other contexts where it is important to understand and apply this relationship. There are many applications to explore in your Biology major – use ChatGPT to investigate these over the upcoming break.This is a very simple problem. It tells us that the rate of change of the volume with respect to time is v’(t) =2t. This means that every second, the volume increases by 2 decibels. The next bit of information given that is important is that we want to find the accumulation of volume on the interval of 0 to 4 seconds. To find the accumulation, we find the integral on that interval. Finding that integral is equivalent to adding the rates of change at each point on the interval of 0 to 4. In the end, we get a total accumulation of 16 decibels over the interval of 0 to 4 seconds.This is a straightforward example problem involving the rate of decibel change and accumulation. Pay particular attention to the rate units (decibels per second) and the accumulation units (decibels). I encourage you to interpret and compare the accumulation in the context of musical composition to accumulation represented as area underneath the rate function’s [v’(t)=2t] graph. Finding and interpreting area under a rate graph for more complicated functions will be on the final exam. Samples from this student’s writing submission confirm many of the quality rankings shared for this AI-SA in the previous blog post. When giving feedback I thought about Clark & Talbert’s feedback loop; the calculus learning objective; and my initial framework rankings. (Clark & Talbert, 2023, p. 12). This feedback loop and my specific learning intentions helped me to focus my feedback. I paid attention to aspects of students’ writing and thinking that went beyond just identifying interesting points or well-worded paragraphs to account for the calculus learning goal and student use of ChatGPT. Based on this student’s performance on subsequent assessments it appears that he used this AI-SA and the feedback I gave him to adequately prepare himself to demonstrate understanding through the remainder of that semester. In closing, consider some takeaways I have been appreciating during this work. First, faculty who want their students to use generative AI in meaningful ways need to be purposeful and detailed about the written expectations they include in their assessments. After analyzing student writing submissions on this AI-SA, I had the distinct impression that students used ChatGPT in ways I desired due to the detailed guidelines found in the assignment itself. I further had the impression that students are ready to step up to the challenges of using generative AI if they are provided appropriate guidance (e.g. ChatGPT-Active and ChatGPT-Inactive components) and scaffolds in their efforts. Finally, the work of creating assessments for learning that incorporate student use of generative AI is no small feat. I suggest that interested faculty start with one assessment and adapt it for student generative AI utilization. Deeply consider student work and feedback, then revise and try it with other students. Email me at atrocki@elon.edu and let me know how it goes. References Boud, David, Phillip Dawson, Margaret Bearman, Sue Bennett, Gordon Joughin, and Elizabeth Molloy, 2018. “Reframing Assessment Research: Through a Practice Perspective.” Studies in Higher Education 43(7): 1107-1118. Clark, David & Robert Talbert. 2023. Grading for Growth: A Guide to Alternative Grading Practices that Promote Authentic Learning and Student Engagement in Higher Education. New York: Stylus Publishing, LLC. Wlodkowski, Raymond J., and Margery B. Ginsberg. 2018. Enhancing Adult Motivation to Learn: A Comprehensive Guide for Teaching All Adults (4th ed). Jossey Bass. Aaron Trocki is an Associate Professor of Mathematics at Elon University. He is the CEL Scholar for 2023-2024 and is focusing on models of assessment and feedback outside of traditional grading assumptions and approaches. How to Cite this Post Trocki, Aaron. 2024. “Utilizing a Framework for Artificial Intelligence-Supported Assessments: Part 2.” Center for Engaged Learning (blog), Elon University. March 12, 2024. https://www.centerforengagedlearning.org/utilizing-a-framework-for-artificial-intelligence-supported-assessments-part-2.