HLTH6055: Leadership and Management for Public Health Improvement is part of an M-level course for qualified health care professionals. This module requires the students to work in teams of ten to explore relevant issues, and a key element of the summative assessment is their performance as a member of the team. The tutor, Yvette Cox, wanted the students to be able to provide each other with detailed anonymous feedback, but the challenge was that there were around 80 students, so a paper-based method would generate 720 (80×9) documents to sort, mark and distribute. ILIAD was asked if an online solution was possible.
Initial discussions centred on the type of feedback that the students would give each other, and it was agreed that rating each criteria as Excellent, Good or Poor would be easier and more reliable than a 1-10 scale. A document Peer review of teamwork was written that clearly described excellent, good and poor behaviours for each of the ten criteria, and this was given to students along with detailed advice about the assessment process. They were also given a Word template to help them make notes that would act as a guide when they came to rate each member of their team at the end of the module.
Students input their ratings using an online survey created using iSurvey. In addition to rating ten criteria for each of their nine team members, they also rated themselves so that they could compare this to their peers’ ratings when they received their detailed feedback at the end of the module. They also needed to add some free-text comments for each member of the team, enabling qualitative as well as quantitative feedback. Completing the survey took students between 20 minutes and 3 hours, with a median of around 1 hour – but this was a major part of the assessment and required a good deal of thought about their responses.
An Excel spreadsheet was developed that would take the raw data from iSurvey and automatically process it so that the tutor could easily print (as PDF files) two sheets for each student; a summary of the ratings that they received for each criteria and a list of the text feedback provided. The tutor also had a summary of all group results and these used automated colour-coding to enable easy identification of both high and low-performing students and groups. Reviewing the results and printing the feedback sheets takes around one day – not bad for 80 students!
In practice, the data often needs some minimal cleaning – typically because a few students have had to leave the course or defer for a year. The spreadsheet has been designed to act intelligently when generating scores for groups that have less than 10 members. The system has been in use since March 2014 and is now well established. The spreadsheet was adjusted for the second run to give greater weight to two key criteria (standard of work and contributing to meetings).
Students have a clear process and criteria to enable them to provide each other with detailed feedback on their team performance. The feedback does not just show average scores, but also the spread of scores; how many times they were rated excellent, good or poor on each criteria.
The tutor has a system that enables the feedback to be gathered, analysed and printed (as PDF) with minimal tutor input and effort. The system automatically highlights students and groups with exceptionally good or poor ratings to facilitate further investigation.
This system could easily be adapted to work in other modules where peer evaluation of groupwork is required. The limitations of the spreadsheet mean that there must be no more than 10 students in each of 10 teams, and no more than 10 criteria which must be on three levels (e.g. excellent, good, poor). Larger cohorts could simply use more than one iSurvey to gather the data.