HomeBlogStudent Voices Student-Reported Benefits and Tensions about Generative AI in Academics: Part 2by Aaron TrockiMay 14, 2024 Share: Section NavigationSkip section navigationIn this sectionBlog Home AI and Engaged Learning Assessment of Learning Capstone Experiences CEL News CEL Retrospectives CEL Reviews Collaborative Projects and Assignments Community-Based Learning Diversity, Inclusion, and Equity ePortfolio Feedback First-Year Experiences Global Learning Health Sciences High Impact Practices Immersive Learning Internships Learning Communities Mentoring Relationships Online Education Place-Based Learning Professional and Continuing Education Publishing SoTL Reflection and Metacognition Relationships Residential Learning Communities Service-Learning Student-Faculty Partnership Studying EL Supporting Neurodivergent and Physically Disabled Students Undergraduate Research Work-Integrated Learning Writing Transfer in and beyond the University Style Guide for Posts to the Center for Engaged Learning Blog In part one of this two-part blog post, I shared some findings from a focus group interview conducted at the end of last semester. In the first part of the focus group interview, students provided their reactions to the Artificial Intelligence-Supported Assessment (AI-SA), Using ChatGPT to Explore Calculus Applications. Analysis of that interview segment revealed general alignment between my initial framework rankings and emergent focus group themes. Students also responded favorably to the distinction between AI-Active and AI-Inactive components of that AI-SA. In the second part of the focus group interview, students considered class survey results related to their experiences using generative AI. I displayed a bar graph that summarized responses for each survey question and then asked the focus group to consider each summary and discuss their speculations/explanations as to why their classmates gave those responses. This structure invited students as partners (e.g. Manor et al. 2010; Cook-Sather et al. 2019) to share and expound on their beliefs and thoughts. Students discussed not only how generative AI was used to complete the assessment but also how generative AI currently is and should be used more broadly in academics. Like the first part of the focus group interview, I audio-recorded the dialogue and had the audio transcribed. I then undertook a qualitative analysis of the transcript to identify themes. This analysis was guided by recommendations from Creswell (2013) to manage data, add memos on the transcripts, and classify the memos under larger themes. Findings from the second part of the interview are shared below, arranged under each survey question’s bar graph summary that students reflected upon. Instances where I asked a follow-up question are noted as well. The first survey item summary reflected students’ higher levels of enjoyment when completing the AI-SA, Using ChatGPT to Explore Calculus Applications. Emergent ThemesSupporting Focus Group QuotesGenerative AI should not be viewed as taboo but as an acceptable tool for learning when used appropriately.– I think that ChatGPT has been kind of a taboo thing in academics where it’s like you’re not really supposed to use it, and if you use it, you’re not supposed to tell anyone. So, being allowed to use it and talk about using it was pretty fun.– I think that the writing assignment paired with kind of filling in the gaps of whatever ChatGPT spat out forces you to think about it [disciplinary content] in a different way as opposed to just doing a problem. I think that it’s a different type of thinking and a different overall experience.Many students enjoy writing in STEM in addition to or rather than completing traditional application problems.– I think that some people just don’t enjoy traditional math assignments, so I feel like if we had some kids in our class that prefer writing over traditional math assignments, they might enjoy this project more. After students spoke to their reasons for the enjoyment levels found in the graph, we moved on to discussing the summary on students’ perceptions of how ChatGPT may have helped them understand mathematics. Emergent ThemesSupporting Focus Group QuotesGenerative AI may be used to produce visuals and examples that support student understanding of concepts and applications.– Using ChatGPT and doing [investigating] topics, real-world topics or real-world situations, gives you a better visualization. Some people are visual learners, so it [visuals produced in generative AI] just helps to maybe solidify certain concepts.– I’m a visual learner, so being able to draw out a graph and then explain it with words was more beneficial for me.Prompt engineering in generative AI can help students explore and understand concepts and applications.– You just keep asking [ChatGPT] different questions until you fully understand it. So, I feel like [with] just repeating that process, I feel like that helped a lot of people understand the concepts that they were focused on. – Being able to ask for sample problems to actually see the math be worked out—even though sometimes it wouldn’t be correct—you would just be able to see how to do the next step in the problem. [This] made it easier to understand when you get the actual problem. Students shared many insights on how ChatGPT can help them understand math concepts, including generative AI for producing informative visuals. We then moved on to address perceptions of how ChatGPT was used in the writing portion of their work. Emergent ThemesSupporting Focus Group QuotesExpectations are needed for students to use generative AI in appropriate ways to support their writing.– It kind of helps me have a general consensus of what to write, but with me personally, I didn’t do [copy] word for word because I wanted to put it in my own words or just paraphrase it.– I just screenshotted some of their answers. I just used those screenshots to say, hey, this is what ChatGPT gave me, this is how I feel about it, or whatever.– It didn’t write it for me, it was still my own words, but I would say that the information that it gave me helped me formulate an idea that I was able to use to put into my own words. – AI is still kind of a touchy topic, so I feel like a lot of students try to not use it. Personally, I’m one of those because academic dishonesty is a very big issue, and I would never want to be in a situation where I’d be accused of using AI to do work for me. After students discussed survey findings related to the AI-SA, Using ChatGPT to Explore Calculus Applications, I shifted the focus group to discussing their use of generative AI in their academics more broadly. Emergent ThemesSupporting Focus Group QuotesStudents are using generative AI in their academics to varying degrees and may be hesitant to share their degree of use.– I feel like a lot of my classmates would say they use it less than once a week because they don’t want to be accused of any academic dishonesty. – It just doesn’t usually cross my mind to use ChatGPT in the first place … So, that’s why I assume that it would be more “less than once per week” than anything.Students often use generative AI as a starting point for individual learning due to the nonjudgmental nature of the technology and quality of information it provides.– My initial reaction to this [bar graph] is I feel like some of it [summarized frequencies of use] might not be true just because what they [students] are saying is the general fear of, I don’t want teachers to know that I’m using ChatGPT all the time, but I’m not using it to get an answer.– I’m more using it when I truly have no idea what’s going on here. Let me ask the bot, and then I can ask follow-up questions without the fear of being judged—because it doesn’t care, it’s not going to think I’m dumb.– I’m using it to understand when I don’t want judgment of asking [questions that reveal my lack of understanding]. The next question I posed to this focus group did not originally appear on the survey that we were discussing. I asked, “Do you think college students should get instruction or training on how to use generative AI such as ChatGPT? If yes, then what should be included in the instruction/training?” Emergent ThemesSupporting Focus Group QuotesGenerative AI is easily accessible and improving, so students need to know when and how to use it in academics.– AI is going to be more popular and more abundant in the future. So, I feel like if there were to be training, it would be more on just how to limit the use of it, and if you were to use it, what questions are you allowed to ask?– I guess to go off honor code because you can’t really control what everybody’s going to do or ask, but there would be training, generally just how to use it properly without just having it do the homework for you.– You could either have it as an elective … or be required maybe not as a full course. You could just kind of be guided on what to do and how to use it, how to use it more intuitively than just tell me the answer—have it spit out stuff. The last question I posed did not appear on the survey. I asked, “How can college professors develop policies about student use of generative AI, such as ChatGPT on graded work, which would be like papers, assignments, quizzes, tests, et cetera. What should the policies be like?” Emergent ThemesSupporting Focus Group QuotesStudents seek guidance on expectations for using generative AI in academics and specifically in each course they take.– I just think addressing ChatGPT in the class and what their thoughts are on it will assure the students of our uncertainty with [what we are allowed and not allowed to do with] it.– On rubrics, teachers should add, “don’t use ChatGPT or use ChatGPT, but only on this part of the assignment.” Or, “you can use ChatGPT to generate a layout of your paper, but don’t use it to write your paper.” So being more clear and upfront about what you can and can’t use it for might be helpful.– I use it as a learning tool rather than a production tool. [implied that instructors should encourage this] When analyzing focus group data and summarizing these findings under emergent themes, I was struck once again by the importance of listening to students. The practice perspective of assessment embraces listening to students to inform teaching, learning, and assessment practices (Boud et al. 2018). This perspective attempts to account for human activities and ways of knowing in particular places and times. Generative AI has become embedded in learning activities for many students and will likely only increase in utilization. Students have much to offer as we navigate our path forward in higher learning. A few themes stood out for further consideration. Students spoke to the current view of generative AI as taboo in academics but also spoke to how it can be used as a productive learning tool. They expressed guidance and clarity from learning institutions and instructions for how and when to use this technology. Generative AI may serve as an entry point for learning as it offers a nonjudgmental platform for asking novice questions. Although students shared potential benefits of generative AI for learning, they expressed concern for overreliance on this technology, especially regarding academic integrity. I encourage you to consider these focus group findings in relation to how you design expectations for student learning and expectations for how students should demonstrate their learning. Disciplinary knowledge should, of course, come to the fore as you consider expectations for appropriate use of any technology including generative AI. Discussing generative AI with your students will promote consensus building on how this technology tool should be used in courses you teach. References Cook-Sather, Alison, Melanie Bahti, and Anita Ntem. 2019. Pedagogical Partnerships: A How-To Guide for Faculty, Students, and Academic Developers in Higher Education. Elon, NC: Elon University Center for Engaged Learning. https://doi.org/10.36284/celelon.oa1. Creswell, John. 2013. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. 3rd ed.Thousand Oaks, CA: SAGE Publications, Inc. Manor, Christopher, Stephen Bloch-Shulman, Kelly Flannery, and Peter Felten. 2010. “Foundations of Student-Faculty Partnerships in the Scholarship of Teaching and Learning.” In Engaging Students Voices in the Study of Teaching and Learning, edited by Carmen Werder and Megan Otis, 3–15. Sterling, VA: Stylus. Aaron Trocki is an Associate Professor of Mathematics at Elon University. He is the CEL Scholar for 2023–2024 and is focusing on models of assessment and feedback outside of traditional grading assumptions and approaches. How to Cite this Post Trocki, Aaron. 2024. “Student-Reported Benefits and Tensions about Generative AI in Academics: Part 2.” Center for Engaged Learning (blog), Elon University. May 14, 2024. https://centerforengagedlearning.org/student-reported-benefits-and-tensions-about-generative-ai-in-academics-part-2.
