Peer assessment: development of experience from e-LATE(D)
Cloud created by:
3 January 2015
“Elearning and Teaching Enhancements (for Social Sciences)” (e-LATE(D)) is intended for social science faculty staff to familiarise them with the techniques and pedagogy of online learning. It is 14 weeks long, and is in its fourth presentation (Jan 2015). Participants are invited to create a learning activity or modify one for online delivery. A link between this and Openpad is now being piloted.
The activity is peer reviewed and then modified taking the peer reviews into account. (We use the term “peer review” to distinguish the less or more formal activity of providing critical comments from the activity of providing a mark.)
Building on the obvious popularity of the peer review approach, the e-LATE(D) team obtained funding under the New Models of Assessment and Tuition Programme to research the experience of the participants and suggest guidelines for future work with undergraduates. Outputs of the research were:
- a literature review
- report on questionnaire and interview data from participants
- practice guidelines
- a suggestion for a pilot at level 1.
The outputs can be found at the e-LATE(D) Peer Assessment home page. (https://intranet7.open.ac.uk/collaboration/strategic-plan/study/assessment/scholarship_projects/peer_assessment/SitePages/Home.aspx)
The research found a high degree of engagement and satisfaction with the peer review process the participants had conducted on e-LATE(D). The experience was reported as thought-provoking, and gave many participants the opportunity to reconsider and in some cases rejuvenate their own teaching practice, as well as gaining new insights into teaching and learning processes, in particular the experience of being a learner. A number of themes emerged:
- uncertainty and anxiety, associated with working outside of the participants’ ‘comfort zone’
- understanding of the need for thorough preparation for both tutors and students, including a clear pedagogical rationale for the process as well as clear practical guidelines for activities
- despite uncertainty, the process was viewed as positive and helpful and developmental, focussing the participants on long term learning goals and on better learning practices, as well as on the subject-based content of the assignment projects
- new relationships emerging between student and student, and between student and tutor, producing the possibility of deeper understandings of the learning as a collaborative process
- greater engagement with the idea of independent learning.
The review covered the pedagogical purposes of peer assessment, questions of design, the effects it has and its effectiveness in enabling independent learning, the validity, authority and fairness of such assessment, tools (intellectual, technological and physical), and possible differences between formative and summative peer assessment.
The questionnaire covered the overall experience of the participants and their experience of different types of assessment, both in the OU and at other institutions; their experience of preparation for the e-LATE(D) peer assessment exercise; their experience of the exercise itself; their feelings of engagement with the task, and its effects on the collaborative process in general. The conclusion is quoted here:
“The responses from our 17 respondents illustrate many of the pedagogic and practical issues that teachers consider in developing and conducting assessment. All were experienced assessors, yet many felt to some degree apprehensive, even scared, by the demands of an unfamiliar process, mirroring what many undergraduate students feel. The importance of good preparation is emphasised, along with a clear framework, supportive oversight by the moderator / tutor, and a system dealing with breakdowns in the process. For many their view of the purpose of assessment was not changed, but their appreciation of nuances and of how good assessment can be achieved was affected. In terms of the wider learning outcome of developing students as part of a learning community, there was some evidence that the peer assessment activities achieved this: (in response to Q 4.1) “I thought it would be a hassle, but it made me care more about eLATE(D) as a whole”.
“These responses will underline our appreciation of both the practical and pedagogic issues surrounding peer assessment and the shape we give to the proposal for a pilot scheme.”
Follow up interviews with a subset of those who responded to the questionnaire covered uncertainty, expertise, being an assessor, being a student, the peer experience, and preparation. We discovered that what being a peer assessor does for the student is central to the process. Several participants referred to learning more through assessing (and through the related practice of monitoring). They acquired a different, more rounded view of what the peer assessment activities and the staff development module as a whole are trying to achieve. The following exchange sums up the process well:
Interviewer: “[H]as your experience of assessment on e-LATE(D) changed the way you approach your own teaching and learning in general ... You’ve put in your comment ‘much more reflective’. You sound like a pretty reflective person actually, so has it really made you much more reflective?”
Participant: “Yes it has; it really did change how I felt about what I do. I have always been careful about how I write [feedback] because as I said I have taken a distance learning degree with the OU myself, but … that was way back and I sort of forgot a little bit what it felt like and it made me revisit that and you know it was really good. It made a huge difference actually”
A three page summary of practice guidelines covers the forms and purpose of peer assessment, preparing the assessment strategy, how to build the marking scheme and how to organise the activity. These guidelines are indebted to RMIT's (2008), ‘Self and Peer assessment’. (http://mams.rmit.edu.au/71ra0k9io8yzz.pdf)
A pilot proposal was produced. The proposal is intended for use in a level 1 social science module but could be adapted to any level and any faculty. The proposal covers the process, the learning outcomes, the marking scheme, tutor preparation, student preparation, and timing of the activity. The proposal included several resources:
- an introductory note for tutors
- sample material for introductory session for tutors
- an introductory note for students
- sample material for introductory tutorial for students (includes separate ppt resource)
- a review framework for students
- example scripts for peer assessment exercise (Example A used in screencast – see 7)
- a review example screencast for students