Nominee
AI in Teaching &
Learning
This project illustrates the use of peer- and self-assessment tools in the seminar Advanced Topics in Machine Learning and Data Science at ETH Zürich. Due to time constraints imposed by the seminar's design, these tools were introduced to promote reflection without altering the course format. Survey analysis shows they supported active learning by encouraging students to critically assess their own work, learn from peers, and better identify areas for improvement, aligning well with the seminar’s learning objectives.
Implementation of the Course
The Learning Objectives provided in the official ETHZ publication accurately describe the main goals of this Seminar.
«The seminar familiarizes students with advanced and recent ideas in machine learning. Original articles have to be presented, contextualized, and critically reviewed. The students will learn how to structure a scientific presentation in English which covers the key ideas of a scientific paper.»
This seminar uses a student-led approach in which participants learn by reading research papers, presenting them in person to peers, and leading discussions. This method builds understanding, communication skills, and confidence while exposing students to diverse presentation styles and ways to convey complex ideas.
The semester-long seminar begins with an introduction, followed by students ranking preferred papers from a curated list of key ML works. Each student is assigned a paper and delivers a 30-minute presentation followed by a Q&A session, with instructors adding context and connections. Two presentations are held per session from early March to late May. Professors meet online with each student twice: first to confirm understanding of the paper’s key contributions, and later to review and give constructive feedback on the presentation. Feedback focuses on improving structure and delivery without reshaping the talk into the professor’s version. Besides, when there are related papers, students are encouraged to discuss ideas with classmates to gain further insights from their classmates’ papers and improve their own presentation and understanding. Finally, the seminar is not particularly affected by the usage of novel AI methods, specifically LLMs. While we assume they use them to accelerate understanding of the papers and to prepare visual material, they still need to fully understand the material and the content they create, and prepare for the presentation. And, for these tasks, AI tools cannot support them.
Therefore, the seminar presents an interesting mix of active teaching, during student-professor interactions, and passive learning, which occurs when students listen to their classmates‘ presentations and later reflect upon them. All this unfolds in a really casual atmosphere, as students are partly relieved during the presentation, knowing that their classmates have faced or will face the same situation. This also promotes the discussion during the Q&A session, where there is always substantial participation.
The assessment is based mostly on the slide deck quality, presentation style, and depth of understanding, with strong performances characterized by clear, engaging talks and meaningful discussions. Furthermore, we encourage a participatory attitude during others‘ presentations and positively evaluate involvement in Q&A sessions.
The described setup is enjoyed by the students, making this seminar particularly well-received. Besides, my experience has always been that everyone’s attitude towards their peers is really positive and constructive, creating a truly relaxed atmosphere. Still, I believe there are some challenges we need to confront, which can be described as follows:
– Students have only one chance to present their work. This does not give them the opportunity to learn from their own mistakes and improve their work.
– After the presentation, students receive no further feedback beyond the numerical grade.
Project Mission
The challenges previously described motivated me to incorporate some exercises into the seminar planning. Specifically, I included peer- and self-evaluation assessments during the course, where:
– All students will have to critically evaluate their classmates‘ presentations using various metrics, including slide quality, presentation style, engagement, etc.
– And they will also have to evaluate themselves at the end of the course before receiving their colleagues’ peer assessments and the official grade.
These will allow students to learn more in detail about their own work through their classmates‘ feedback. All these will be lessons learned for future presentations, where they can implement measures to address weak spots. More importantly, through an honest exercise of self-evaluation, they are encouraged to reflect on their own work and identify areas for future improvement. I believe these measures help resolve the challenges described before.
Effects on Student Learning
The exercises proposed improved the following teaching aspects:
– Engagement: by having to provide feedback on their peers’ presentations, students had to pay more attention and analyze in more detail some specific aspects of the presentation. This helped them engage and reflect upon details they might not have been aware of without this exercise.
– Critical thinking: first, the peer-evaluation exercise forced them to engage more actively during others’ presentations, in order to provide well-grounded, justified feedback. Furthermore, they had to do a similar reflection exercise to assess their own work.
– Self-criticism: during the self-evaluation, they had to specifically consider which aspects they could improve, especially compared with their classmates’ presentations.
I could validate these aspects by comparing the peer- and self-assessment surveys. The analyses showed an interesting alignment, underscoring that students can:
– Provide insightful, useful feedback to their classmates, highlighting relevant areas for improvement.
– Reflect on their classmates’ work to identify their own strengths and weaknesses.
– Identify the most important areas for improvement in the future.
Finally, the feedback from a final survey I sent to the students helped me validate the usefulness of the exercises. All the students who answered the surveys responded positively to the self-assessment, demonstrating that when encouraged to do this type of exercise, students perform a genuine and useful act of self-learning.
Additional Studies
Furthermore, I had a longer-term vision for these exercises, as I wanted to test some hypotheses through the surveys to better understand how effective the learning experience is. These hypotheses are:
1. Is there an indirect learning process occurring throughout the course? Where students presenting later in the semester benefit from the good and bad practices seen in their classmates’ presentations.
2. Do the students under- or over-estimate the complexity of carrying out the work of preparing and presenting a scientific paper?
3. Are the students capable of doing a serious and honest exercise of self-criticism? And, can the students identify and provide accurate feedback?
I provide brief answers to each of the previous questions, accompanied by figures to support them.
1. As shown in Figure 1, there is a significant increase in the overall score on the peer-assessment questionnaire. This improvement is more pronounced in specific rubrics such as “Engagement and Enthusiasm”, as illustrated in Figure 2. This illustrates that there is some degree of active learning throughout the course, where students also learn to prepare better presentations by understanding the strengths and weaknesses of their classmates’ presentations.
2. As illustrated in Figure 3, after their own presentation, the average score provided by students in the peer-evaluation questionnaire increases. This can be explained by the fact that students may tend to overestimate the complexity of preparing and presenting scientific material, and only when they present themselves do they realize and adjust accordingly.
3. After summarizing the feedback provided in the peer- and self-evaluation questionnaires, we observe a noticeable overlap between them, which validates that students can both provide insightful, well-grounded feedback and also carry out an honest self-evaluation.
Figure 1: Rolling window average (n = 5) for overall score
Figure 2: Rolling window average (n = 5) for engagement and enthusiasm
Figure 3: Violin plots comparing score distributions before (left) and after (right) each student’s own presentation date
Outlook And Future Improvements
Based on the analysis of previous hypotheses, I observe that the main caveat of the current seminar structure is that students who present later in the semester are more likely to receive higher grades, due to learning that occurs throughout the semester. This phenomenon may be aggravated in seminars with fewer sessions, as students will have less time to reflect on their classmates’ presentations, benefiting mostly those presenting on the last day. To curb this, I propose the following two ideas to improve the seminar structure:
– A seminar divided into two phases: the first, with a structure similar to the current one, but with the idea of doing a test run of the presentation. The collected feedback will be used in phase 2 to improve the slide deck and presentation style. Only the latter presentation will be evaluated. However, implementing this structure will require additional sessions, a limiting factor considering the number of credits assigned to the seminar
– A session to expose the students to common good and bad practices. By showing these well in advance, for example, through exaggerated mock-ups or video examples, they will be more aware of these problems and consider them when presenting their work. This, at least, will help address the disadvantage that the first students face, namely, a lack of sufficient examples and feedback.
ETH Competence Framework
-
Social
Competencies -
Personal
Competencies
– Communication: supported by the seminar itself, as it is one of the main learning objectives.
– Critical thinking: the peer-assessment questionnaire encourages students to engage actively during the presentations of their peers, instead of passively listening to them. They need to critically think about the strengths and weaknesses of their classmates’ work.
– Self-awareness and self-reflection: the self-assessment questionnaire supports these competencies by prompting students to reevaluate their own work after attending all the other presentations. Only then can they be aware of their own strengths, and also aspects to improve in the future.
Which Elements of Your Project Would You Recommend to Others?
I would recommend to all lecturers the application of peer- and self-assessment exercises. They are easy for the lecturer to implement and do not require a substantial time investment from the students. Even though peer-assessment is not always possible, i.e., there is no student participation, self-evaluation could still be put in place to encourage students to reflect at the end of the semester about, for example, the most challenging parts of the course, material that requires further understanding for upcoming courses, etc.
