One-to-one Progress Meetings with immediate, individual Feedback

Overview of the Course

What is the subject context of the course?

The goal is to learn important basic computer science concepts and techniques for the implementation of interdisciplinary programming projects. The course is based on a problem-based blended learning approach that prepares student in a structured way to solve programming projects independently (more details see below).

Main objective of this course: What should students learn and be able to do at the end of the course? 

Students learn key programming concepts and transfer them to practice and to their subject area (e.g. life sciences). The focus is not only on writing code, but also on discussing and further development of existing code and algorithms.

Students learn…
… how to encode a problem into a program, test the program, and correct errors.
… to understand and improve existing code.
… to deal with the complexity of real data.
… to store data in a suitable data structure.
… to query databases and understand and evaluate the corresponding database model.
… to implement mathematical models as a simulation.
… to explain and apply standard algorithms.

Why was the specific assessment format chosen? 

The assessment format is designed to achieve the following three objectives, in line with the problem-based blended learning format of our courses:

1. Regular individual progress reviews instead of summative group exams

Regular feedback shows students where they are in the learning process at a time when they can still adjust their learning behavior (status assessment). Our system can adapt to different levels of knowledge. There are groups of students who have never programmed before, others who need to refresh their knowledge and there are those who are already advanced programmers. In this system, one group is not overwhelmed while another is bored. Everyone progresses at their own level.

2. The focus of assessment is on understanding concepts rather than completing assignment products

Instead of grading hundreds of programming assignments, our teaching assistants provide personalized oral feedback in regular one-to-one progress meetings. Since the focus of progress meetings is on concept understanding rather than on submitting a perfect final product, students are encouraged to engage with key concepts regardless of the tools they use. AI tools, which are widely used in programming nowadays, are not prohibited. However, they should be used in such a way that conceptual learning can still take place. Our policy is that students must always be able to defend their solutions.

3. Making mistakes and misunderstandings are an important part of the learning process and are clarified during individual coaching

This formative, ungraded assessment promotes an open learning environment free from the pressure of exams. Misunderstandings and misconceptions are addressed individually, allowing students to learn and grow without fear. This role is highly appreciated by the teaching assistants, and it helps them develop their own transferable competencies.

How are students prepared for the assessment?

The course is based on a problem-based blended learning concept. Exercise groups and lectures are largely replaced with student-centered instruction using tutorials and videos directly within a web-based programming environment (Code Expert and E.Tutorial, a winning project of the KITE Award 2018). These videos and tutorials introduce the key concepts, terms, and techniques and demonstrate how to solve a project directly in the programming environment. After this guided part, each of the 6 modules contains an independent project phase with an additional task, which is then discussed in the progress meeting.

The goal is to create a learning environment that fosters an open exchange with teaching assistants on equal terms. For this to succeed, the teaching assistants need to be well prepared.

Shared Experience

How many times has the assessment been conducted in this format?

In a previous format for almost 20 years! At that time, we started with an analog numbering system in a computer room to call up students for their exercise presentation. The Innovedum project “Personal Electronic Learning Environment” PELE (2011-2014) provided the foundation for the automated exercise group management. From there on, we have been holding progress meetings in the described format with consistently very good student feedback. Over the last 10 years, 10’092 students have been assessed up to six times per semester by 480 TAs through progress meetings in 27 courses. During COVID-19 pandemic, we created the option for online progress meetings with Zoom. In 2021, we migrated the functionalities of PELE to Code Expert and we supported Sudret & Lüthen from D-BAUG in assessing their lecture “Programming for Engineers” (101-0700-00L) with regular progress meetings using Code Expert.

What contributed to the success?

The assessment format is very popular with both students and TAs. Students learn in a more self-directed, motivated, regular way and, above all, not just to pass an exam. Regular personal and constructive feedback from the TAs builds students’ self-confidence, an important prerequisite for finding fun in the subject (like programming in our case).

The assessment format can be scaled up well to large student groups. Once it is up and running, it is autonomous and decentralized and no longer needs to be motivated and driven by the lecturers. This is not only an enormous benefit for the lecturers and everyone’s workload is better spread over the semester. Many of our students report that they do not have to do much repetition for the final exam as they have done this consistently throughout the semester.

Success depends on several factors, the most important are:

• Learning material that enables students to solve projects on their own and supports conceptual learning:
  One of the most important prerequisite for this assessment format is having interesting and feasible project tasks, which form the basis for the student’s performance and the progress discussions. Ideally, they relate to the student’s degree program. This also includes structures that help them to recognize and name the underlying theoretical concepts. In order to prepare students conceptually for solving the project tasks independently, we have developed detailed online tutorials (E.Tutorial).
• Efficient software for exercise group management:
  An efficient software solution is needed to organize and schedule the progress meetings between TAs and students, especially for larger cohorts. The presentation mode of Code Expert enables enrollment, registration, calling and peer evaluation of the progress meetings (Code Expert guide).
• Focus on TA recruitment, training and feedback:
  This assessment format uses the resource of TAs by giving them a new role as student coaches. This role requires a set of communication skills and attitudes that we do not assume and that can only be taught to a limited extent. This is why we invest a lot of time in recruiting, training and feedback for TAs, particularly at the beginning of the term. Our courses offer not only students but also TAs the opportunity to acquire transferable skills (see section «Shared Materials»).
• Attractive bonus system:
  If attending progress meetings is included in an attractive bonus system, it creates an incentive for students to participate.

What were the challenges and how were they overcome?

There are several challenges that need to be considered when introducing one-to-one progress meetings:

• How do TAs learn effective questioning techniques and to give high quality feedback?
  We organize a workshop for all our TAs at the beginning of the term. There we do video case study discussions and role plays, in which one person acts as a student, one as a TA and one as a observer of the progress meeting (see section «Shared Materials»).
• Dealing with role changes:
  The introduction of project based learning with regular progress meetings creates possible role changes of the whole teaching team. As a lecturer for example, you are no longer the center of teaching. As the primarily responsible person, you only have an indirect influence on the students’ learning behavior. It is also possible that the introduction of the problem-based approach with progress meetings results in fewer students attending lectures, which does not mean that they are not active.

Are there any further developments planned?

The following further developments are planned:

• Development of a new 1 credit point coaching course for TAs with focus on communication and feedback skills.
• Replacement of graded written examinations with new problem-oriented examination formats with individual review and feedback (e.g. as ungraded semester performance).
• Development of a new on-site format to replace the regular lecture.

What tips would you give lecturers who are planning a similar assessment?

The introduction of regular progress meetings can be more than just the implementation of a specific assessment format. The changeover is feasible and extremely worthwhile.

We are frequently asked the following questions regarding the introduction of progress meetings:

• How many times should progress meetings be held in a course?
  It is important to repeat the progress meetings several times during a course. We believe that a two-week interval is ideal, resulting in 6 to 7 progress meetings per semester. For a first run, three times is a good, feasible value.
• Should progress meetings be graded?
  We do not recommend grading the progress meetings, in line with the literature. Grading would negate many of the positive effects of this assessment format, which focuses on improving the learning process and conceptual understanding.
• Can feedback also be provided by peers?
  It proves to be very challenging assessing someone’s conceptual understanding from a conversation about a solved project. We do not believe that peers can do this to the same extent as experts or, in our case, ex-students who we train ourselves.
• Group progress meetings instead of one-to-one meetings?
  We recommend 1:1 meetings, especially for foundation courses. If group work is an explicit aim of a course, group progress meetings may be an option. However, it should be noted that it can be additionally difficult to give feedback to individuals within a group.
• What are the first steps you recommend?
  Start with a few projects (e.g. three). Create materials (tutorials, videos, etc.) to prepare students conceptually for solving the projects independently. Let the project results be discussed in 1:1-progress meetings and prepare your TAs to do progress meetings.

We are happy to support you if you are planning a setup for one-to-one progress meetings with Code Expert. Write to: expert@ethz.ch