Generation and Curation of Multimodal Learning Resources

Implementation of the Course

The project has been implemented within a blended teaching environment, where lectures are delivered in person and learning materials are distributed via digital platforms. Although detailed lecture notes are highly beneficial for students, the time required for instructors to produce such documents is often prohibitive. This initiative addresses this challenge by enhancing the post-lecture learning experience, providing students with AI-generated materials that surpass conventional note-taking and slide reviews. The implementation follows a structured workflow initiated during each lecture session, with audio captured using a dedicated voice recorder to ensure high-quality sound and accurate transcription. After each lecture, the audio is processed through AI transcription tools to generate text versions of the spoken content. These transcriptions are integrated with lecture slides and relevant reference materials, including journal articles, textbook chapters, and supplementary documents, and then uploaded to generative AI platforms. The AI platforms produce various types of learning materials tailored to students‘ needs and preferences. Detailed lecture scripts capture the narrative flow and key explanations from each session, preserving the lecturer’s chosen logical progression of concepts. Podcast-style audio content offers students a flexible, engaging format for reviewing material during commutes or other activities. This multimodal approach accommodates diverse learning habits, enabling students to engage with course content in multiple ways and at different times. A critical component of this implementation is the rigorous review process conducted before releasing learning materials. Each piece of AI-generated content undergoes thorough examination and revision by the instructor to ensure scientific accuracy, correct errors, clarify ambiguous sections, and confirm alignment with course learning objectives. The instructor refines the content to enhance student understanding, rather than merely duplicating the live lecture. This human-in-the-loop methodology maintains pedagogical integrity and ensures that AI serves as a tool for enhancement rather than a replacement for instructor expertise. Students access the learning materials through the Confluence platform at ETH Zurich, which includes a comment section for asking questions or engaging in discussions. This asynchronous availability allows students to engage with the content at their own pace and revisit challenging concepts as needed. Preliminary observations suggest that the availability of comprehensive learning materials has improved student preparation, though analysis of performance metrics remains to be done. Informal student feedback indicates that lecture scripts help identify gaps in understanding, and the podcast format supports both lecture preparation and content revision for the final exam. Several challenges have arisen during implementation, including initial transcription quality varying depending on the audio capture hardware, the AI platform used, and the system prompt structure. The AI platform occasionally misinterprets technical terminology or fails to capture nuanced explanations, highlighting the importance of careful instructor review. The time required for quality control is substantial, ranging from 2-3 hours per lecture hour, but decreases substantially as the instructor develops efficient review workflows and templates for the generative AI model. A key lesson is that transparent communication with students about the process is essential. Students value knowing that materials are AI-assisted but instructor-verified, which maintains trust in content quality. Another insight is that the materials are most effective when explicitly integrated into the course structure rather than offered as optional supplements. The project demonstrates that AI can serve as a productivity tool for lecturers, augmenting instructors‘ capabilities without replacing human judgment. Time-consuming and repetitive tasks, such as transcription, initial content structuring, and format conversion, are delegated to generative AI while the instructor retains full control over pedagogical decisions, accuracy verification, and content refinement. Students are informed about which materials are AI-generated and the review process, promoting transparency and fostering an understanding of appropriate AI use in academic contexts. This balanced approach models responsible AI integration and delivers tangible benefits to student learning.

Motivation, Project Mission, Vision Statement 

Students often struggle to capture complete lecture content while engaging and participating in frontal lectures, and traditional lecture slides frequently lack the narrative context necessary for effective learning. Instructors also face time constraints that make it impractical to produce comprehensive lecture notes, despite their pedagogical value. This project addresses these challenges by developing a sustainable workflow that combines AI efficiency with rigorous instructor oversight to generate high-quality learning materials from audio recordings. The objective is to provide students with comprehensive, accurate, and pedagogically sound resources that support diverse learning preferences while maintaining academic integrity through careful human review. This ensures that AI serves as a tool to enhance, rather than replace, instructor expertise. This approach presents a new strategy for pre- and post-lecture learning support, enabling students to access detailed materials that capture both content and pedagogical narrative. As a result, students can engage more deeply with course material at their own pace, while instructors redirect time from manual documentation toward higher-value activities such as updating lecture materials and providing personalized student support.

Innovative Elements

This project introduces three key innovations in teaching practice. First, it systematically integrates AI transcription and generative AI platforms into the post-lecture workflow, automating the production of learning materials that would otherwise be prohibitively time-intensive for instructors to create manually. Second, it transforms single-format lecture content into multimodal resources, including detailed scripts and podcast-style audio, accommodates diverse learning preferences, and enables flexible review schedules. Third, it establishes a quality-controlled AI integration model that maintains academic rigor through thorough human review, demonstrating how AI can augment rather than compromise pedagogical quality.

Effects on Student Learning 

Effects on Student Learning
Informal student feedback indicates that the AI-generated learning materials significantly enhance their ability to review and understand course content. Students particularly value the multimodal format, noting that different material types serve different study contexts, with lecture scripts used for focused study sessions and podcasts for commute-time review. While formal comparative assessment data is still being collected and analyzed, preliminary observations suggest improved performance on exam questions that require synthesis and application of lecture concepts.

Which Elements of Your Project Would You Recommend to Others? 

The key elements of this project are highly transferable and adaptable to other colleagues’ needs. Audio recording hardware is affordable (CHF 40-100) and refined system prompts are readily available. With current automation tools, enhanced reasoning quality of LLMs, and AI agent orchestration strategies, the time required to create learning materials with high pedagogical quality can be significantly reduced.