Me and my colleague Prof. Randi Karlsen have performed mutual peer audit of our teaching (Aalberg 2009). The peer audit has been performed over two periods in the autumn of 2014 and the autumn of 2016.
Peer audit organisation
The peer audit has been done when we both were teaching the course computer communication and security. In each period we did peer audit of three lectures each. Each time we had a focus on one specific topic regarding teaching. In 2014 the three topics were:
Usage of presentation materials (PowerPoint or similar)
Detailed example walk-through
The overall lecture
We both were observed and were the observer 3 times, once for each topic. In 2016 the three topics were:
The usage of figures and illustrations in the lecture
Live-coding at lecture / student presentation at lecture
The overall lecture
Also this time we both were observed and were the observer 3 times, once for each topic. However, this time we had a different second topic. I did a lecture with live-coding (live development of a program) and Professor Karlsen organised and managed a series of lectures where students presented a given topic (we only did the peer audit for one of the lectures in this series).
We organised the peer audit in three phases: pre-lecture, observation, and post-lecture (Aalberg 2002; Lauvås and Handal 1990). In the pre-lecture phase we met and discussed the topic. The focus in the meeting was how this topic could contribute to the teaching of the learning goals in the selected lecture. After the specific learning goals were listed, the lecturer presented how the given topic would be used to improve the lecture og contribute to the students’ achievement of the learning goals. When the topic was the overall lecture, the lecturer presented how each stage of the lecture was planned and the thinking behind it related to the learning goals. During the observation phase the observer tried to evaluate the lecture based on the topic and the planned learning goals. The behaviour and respons from the students were also observed and used in the evaluation. In the post-lecture phase, the observed lecturer and the observer met and discussed the lecture with a main focus on the selected topic. We tried to organise this as a discussion and not a presentation of an evaluation report.
Lessons learned from peer audit
The peer audit produced new insight into teaching both for the observed lecturer and the observer. Being the observer is surprisingly educational. Doing planned observation of a colleague teaching makes you reflect over your own teaching. It is also surprisingly difficult to focus on observing (only) the selected topic. We decided early on to allow the observer to also give feedback on other things observed in the lecture, but we tried to keep the main focus on the selected topic.
Being observed when lecturing was uncomfortable the first time, but later you forgot that you were observed by a colleague. In the beginning this might have influenced the lectures. If you are conscious about being observed you might change your behaviour. We discussed this, but we could not conclude that we observed a uniform change of behaviour. For instance, one of us felt an increase in speed when being observed, but the other one felt a decrease in speed when being observed.
The pre-lecture meetings were short without too much time for discussions. The main focus was to let the lecturer present his and her thoughts on the topic related to the learning goals and how this was planned in the lecture. During the lecture the observer did take notes regarding the performance of the lecturer and the participation of students. In the post-lecture meetings both the observed lecturer and the observer presented their view on how the lecture went regarding the topic and the learning goals. Most of the time these two views coincided with each other, but a few times the observer had a different view on how something were presented to and received by the students.
Regarding the 6 topics selected, this is what we learned through the peer audit of each other:
The usage of PowerPoint (or similar presentation materials) makes it possible to include more learning goals in a single lecture and it could help students organise their thoughts on, and relation between, what they have learned. However, PowerPoint might passivate the students. It is easy to loose their attention and sometimes the details are lost in the head-line like approach of such presentations. It is also a lot of work to create good presentations that is balanced regarding overview and details.
The walk-through of a detailed example can give students a deep insight to a problem and its solutions. We observed that for this to work well the students should already have a good understanding of the problem or its context. Otherwise, all the details become overwhelming and difficult to grasp. If the students are not able to follow the example (too fast, not enough background, or a minor slip in their attention) the detailed walk-through is a waste of time for them. It is important to keep the attention of your students throughout the example either by involving them (what is the next step) or repeating steps when you feel you have lost them. A major draw-back of detailed example walk-through is that it can be very time consuming. It has to be used with moderation.
The usage of figures and illustrations can help students understand important part of the learning goals. One main observation was that a good way to use illustrations when explaining complex matters regarding computer communication and security (protocols) was to start with a simple overview figure first, and then reuse it with added and removed details for different purposes. For example, to have a figure with an overview of all protocol layers in a protocol stack, and then zoom in to each layer an expose the details regarding specific layers later in the same lecture. In this way the students learned the connection and the big picture of the protocol layers and saw that in relation with the details of each layer. Good figures and illustrations will make it easier on quicker for students to grasp some learning goals, but good figures and illustrations are time consuming to produce. Bad figures (for example with too much details) can be confusing and give little new insight to the students.
Live-coding is an approach to teaching computer science by developing code (program or software) live during the lecture. It is a common approach when we learn students to program since we demonstrate all the detailed steps they have to perform. The challenges is that it is easy to introduce errors that are difficult to find in a (stressful) live setting. It is also very time-consuming. Live-coding is a version of walk-through of detailed examples, and it has similar characteristics. However, coding is the most important tool of the computer science students and it is by definition a precise language to express solution to problems discussed in the course. It is also very motivating for the students. We observed that they payed strong attention and were very involved during the live-coding in the lectures. Some of the students are also good in finding errors and suggesting improvements during the lecture. Live-coding also gives the students direct feedback on approaches to solve a problem: we can test that the code written actually work by executing the code. And if the code does not work as intended, we can improve and retry it.
In lectures with student presentations, the students are given a topic from the syllabus to present for the other students. We experience that the students presenting will gain deep knowledge of the topic they are given. The other students can also be more engaged when a fellow student is presenting a topic, but this is not always the case. We observed that when one or two students start to ask questions to the presenting student, other students will follow with their own questions. As a result of this observation we appointed two fellow student to a specific role for each student presentation. Their role were to prepare and present at least two questions each to the student presenting. This seems to initiate more lively discussions at the end of each presentation.
The last topic we had both in 2014 and in 2016 were the overall lecture. When planning, we felt that we should at least once (each year) have focus on the overall lecture. However, our experience is that it is difficult to avoid discussing the overall lecture every time. So the lessons learned presented here are based on the peer audit of all lectures. What I present here is also based on our continuously discussion regarding teaching from 2014 and until present. Somethings learned are also discussed in more details in other parts of this teaching portfolio. To summarize the more general points from the peer audit of the overall lecture:
A good lecture is a well-prepared lecture. This is obvious, but the observations during the peer audit clearly confirmed this. When the lecturer felt that he or she could have been better prepared this was easily observed by the observer. Either by the lack of good presentation materials, uncertainty during the lecture, missing structure during the lecture, or bad timing of the lecture.
A topic of interest for the lecturer is well taught. When the lecturer finds a topic interesting, he or she is engaged during teaching. This makes the students engaged. Both the observer and the observed lecturer felt that lectures gained from this.
We both learned that sometimes as a lecturer you present material to students that can only be understood in a given context, but you have forgotten to present the context or your presentation of the context was too weak.
Aalberg, Thore K. 2002. Individuell Veiledning. Fagbokforlaget.
———. 2009. “Kollegaveiledning I Høyere Utdanning.” Nordic Studies in Education 29 (2): 185–93.
Lauvås, Per, and Gunnar Handal. 1990. Veiledning Og Praktisk Yrkesteori. Cappelen.