Advancing Data-Driven Learning with Generative AI: Leveraging LLM and RAG for Enhanced Analysis of Research Article Introductions

This study explores the application of Generative AI using Large Language Models (LLMs) to perform move analysis on a dataset of 20 research article (RA) introductions. By identifying key linguistic features and annotating these texts, we incorporate Retrieval-Augmented Generation (RAG) technology to create a dynamic database that facilitates an innovative corpus lookup system. This system allows student users to search for specific linguistic constructs within RA introductions efficiently. We quantitatively compare the effectiveness of this LLM-empowered corpus lookup with traditional concordance line lookup methods commonly used in data-driven learning. Our findings indicate that after implementing few-shot prompting techniques, the interrater agreement between human annotators and the AI predictions exceeds 90%, showcasing a significant alignment in linguistic feature identification. Additionally, a live demonstration of the system is available, illustrating its practical application and ease of use. The integration of LLM and RAG technologies not only enhances the searchability and relevance of educational materials but also significantly improves the learning process by providing precise and contextually appropriate results. This approach promises a substantial improvement over traditional methods, making it a valuable tool for educational advancements in linguistic research.