Cloudworks is no longer accepting new user registrations, and will be closing down on 24th June 2019. We hope to make a read-only archive of the site available soon after.

Thinking about assessment practices

Associate lecturers' National Conference,

Cloud created by:

Gráinne Conole
21 November 2009

This cloud has been set up to support the assessment session at the Open University associate lecturers' national conference on the 21st November 2009.

Questions

  • 1. Do we over assess? From a student perspective, from a tutor perspective?
  • 2. How do we build in variability? Alternatives to 3000-word essay
  • 3. Right ratio of TMAs to ECAs to exams?
  • 4. What is the role of self-assessment, formative & summative assessment? 
  • 5. How do we assess beyond the individual?
  • 6. Can we move to move standard guidelines for marking criteria/what are appropriate marking criteria
  • 7. Should we move to awards-based assessment? If so, how?

Extra content

Some definitions:

  • TMA Tutor Marked Assignment
  • ECA End of Course Assesssment
  • NSS National Student Survey
  • CETL Centre Exellence Teaching Leaning
  • S104 is one of our first year Science courses.

 

Gráinne Conole
12:30 on 21 November 2009

The following are some notes from the plenary general discussion, around the 7 questions posed ahead.

  1. Do we over assess? What is assessment for? To get the students to improve based on feedback – if they read it!!! Focus on the mark!! What would be the impact of delaying the time between giving feedback to students and the actual mark? Would this then result in  students having to engage with the feedback more? There is a role for more frequent assessment
  2. How to build in variability. There is already a lot of variability, not many of us teach courses where it is only 3000 word essay and where it is that it was the right assessment for that course. We felt overall the assessments were correct for the course. It’s up to the course teams role in terms of the way they design it.  Maybe we should incorporate ALs more in the design process? We felt that not all course teams do test the learning outcomes out enough… In terms of course design, it might be useful to consider designing course, which had a bank of learning objects from which individual tutors could then choose a selection. Alternatively the choice could be made by the students themselves, this would be potentially a way of giving students more choice and flexibility
  3. What are the roles of TMA? There are a range of issues about the TMA deadline and the impact this has on the students and their motivation. It’s important to recognise that TMAs and deadlines do motivate students and help to keep them on track with the course. Openings courses have a different model (they have two formative TMAs) and they work well for students at that level. There was some discussions about iCMAs and what they can do in place of TMAs. There was some discussion about the balance of lots of little assessment compared with fewer larger assessments and the pros and cons of each. Can we find more ways to give students more opportunities to go back and engage in reflective learning. For example what if the next TMA was around rewriting the essay from a previous TMA based on what you know now (and based on the feedback received), concentrating on what would make it better. Exams: general consensus was that we are fans of exams in maths and science. Worried about fear of exams and issues around hand writing. Balance of TMAs and ECAs/exams – you can get high marks on TMAs but can be slashed in a bad exam mark. There was a feeling that students tend to ignore study guides to concentrate on TMAs. On the main the group was in favour of projects. The Science course S104 has a lot of short iCMAs, which works well there but would it work in all subjects, for example where you might want to build up a bigger picture first. It was felt that P/G level is different and therefore the types of TMAs are likely to be different.
  4. Self assessment is good on the whole.  What about the role of peer assessment. Will students do formative assessment? Self-reflection elements in the assessment questions would be useful. One participant gave a nice example of a peer assessment activity, it is an online exercise students in pairs and then pass over and comment on each others’ work – helps develop their critical abilities. Also made them realise how difficult it is to mark and to not give negative comments
  5. Assessing beyond the individual: The group discussed the issues around trying to assess both the team work and the quality of the final product. We did feel that the assessment should include a self-assessment, stating what they had contributing.  The group discussed the difficulty of awarding individual makers around group tasks, how do you differentiate? There are also issues in terms of tutor workload for this in terms of facilitating the group and a new skills. If it is made a learning outcome around collaboration is this an issue.
  6. Marking criteria even with tight criteria there will be differences – it’s about professional attitudes. One idea was that at P/G student project negotiation can be very powerful. If extensive guidelines are given to students, do the tutors actually need any additional guidelines and isn’t there a merit in making the guidelines as transparent as possible anyway? Self-assessment at p/g level is possibility even more important, looking beyond the course
  7. Awards based – there is a case for overarching assessment for some qualifications – particularly in professionally-based/vocational courses. Did feel that each course should be assessed on its own merit. Should there be an overall examination type assessment, if contributing courses were only ECA based.

Gráinne Conole
15:52 on 21 November 2009

Indiviudal group summary notes:

  • Do we over assess? What's assessment for? Student improvment based on feedback (if they read it!); pacing students through the course, making them learn/
  • Could we do it by formative assessment? Will they do it?
  • How do we build in variability? Do we need it? That is something that is traditionally done by the course team not the ALs. How do we build in working collaboratively? Important to note that the 3000 word eassy is appropriate for some courses.
  • Assessment must be of the appropriate length and number for the tutor/student/course.
  • Assessing beyond the indiviudal - group tasks, wiki shows who's done which work, assessment should include self-assessment by each student of how they've contributed. Importance of knowing who has done what. Need to be able to assess both the teamworking and the quality of the final product.
  • Marking criteria - even with tighte criteria there'll still be differences between markers.
  • Students often just look at the mark they received and don't read the comments. What can we do to encourage them to take more note of these?
  • Awards-based assessment - each course should be assessed on its own, there may be a case for overarching assessment for some qualifications. Should there be an overall exam if the courses are all assessed by ECA?

Gráinne Conole
15:59 on 21 November 2009

Group discussion notes

– Do we over assess?

a) from a student workload perspective?

  • We can over assess for example a 60 point course might have 7 TMAs, a number of iCMAs, a collaborative project and an end of course assessment.
  • The purpose of assessment is to get students to improve based on the feedback they receive. But it was felt students don’t engage with feedback as fully as they might.

2 – Given the range of learning outcomes that we are addressing, the pedagogies we are using and the learning activities that students undertake how do we build in appropriate variability in the types of assessment? What are the alternatives to the 3000 word essay and how should we be using them?

  • There is variability – if it was on the course then right assessment. Part of design of the course. ALs should be used to input to course design especially around assessment.
  • Course design – provide more learning objects – to offer more student choice.

3- Have we the right ratio of TMAs to ECAs to exams? What would be the arguments for increasing or decreasing the current ratios?

  • The only motivation for students is TMA deadline, self or formative assessment wouldn’t provide this motivation. However on Openings courses the two TMAs are formative (students have to submit one to enable them to take the ECA). And they have no fixed deadline. So tailoring assessment to level and skill of student is important. MCT use early assessment to encourage students to get to engage with the course material. Teaching is in lock step with assessment so opportunities to change assessment are limited. There could be an adverse effect on retention if number of TMAs were reduced. CMAs have some similar properties in engagement, pacing, providing feedback so we don’t need to rely solely on TMAs. There are differences between subject areas – eg iCMAs might work better in Science.  If students do not submit a TMA it is often too late to pick up the problem. Looking at online engagement to pick up student progress might be useful. There is a balance  to be made between lots of little assessment points vs fewer larger assessment points. How students study (eg cramming method) plays into the design and development of effective assessment strategies. The group felt that students are unwilling to read the assessment guidelines. Students want to submit drafts – should we facilitate this and if so how? Assignments where students have the opportunity to engage in reflective leaning would be valuable, as evidenced in AA100.
  • There was acceptance of the value of exams in maths as a defence against plagiarism/ false identity. Some courses require an exam as they are part of externally accredited degree. Fear of exams is strong: handwriting is a problem, time pressure, poor memory, the surprise question, luck of the draw what I have revised, fear of/ history of failure. Providing opportunities to engage in mock exams (eg in tutorials) may reduce fear. Design exams to overcome these problems eg open book. ECAs and exams test different skills. It was noted that GCSEs and  A levels are going back to exams.

5 – Should we move towards award based assessment, and if so how?

  • For some qualifications it would be useful to check achievement. Each course is currently assessed on its own. There should be an overall exam if all courses in award had ECAs.

4- What is role of self assessment, peer assessment ; formative vs summative assessment

  • For science, maths, computing iCMAs can be useful. They can be useful in other disciplines such as social work if students need to understand factual data.
  • iCMAs can be fun.
  • There was a general perception that students don’t want to know about formative assessment. Though openings courses provide an exception to this.
  • Adding self reflection elements in assessment questions would be useful, but we need to convince students of the value of this.
  • Peer assessment with students commenting on others work provides a good exercise and develops students’ critical abilities. It also shows students how difficult it is to mark and come up with positive comments.

5 – how do you assess team-based activity/ beyond the individual?

  • How do you award an individual mark based on taking part – is it all or nothing? What does assessing the quality of an individual’s contribution mean? There are difficulties in putting collaborative work in the first assignment eg with delays in some students getting equipment (esp FAF and LEQ). If teamwork is requires who puts students into teams and on what basis? It was felt that students were reluctant  to engage in group work. For students in prisons there were additional difficulties in collaborative activities though some examples were given of how this works. There was some concern about the amount of tutor time taken to make it group work a success. Course teams need to consider how hard it is to run/ mark group work. Including group work was considered a challenge at level 1. Students were concerned about the perception of the level of ICT skills required. The solution might be to introduce an introductory ICT course, this already existed as an extension to an openings course. There is a bootstrap problem with ICT skills if we expect students to go online (eg through openlearn) to pick up these skills.
  • Self assessment could be included on a student’s contribution to the group.

6 – Can we move to standard guidelines for marking criteria/ what are appropriate marking criteria

  • At PG level student project negotiation is important. Self assessment at PG level is more important especially as students will be expected to look outside the course material.

Peter Wilson
16:40 on 21 November 2009 (Edited 08:49 on 22 November 2009)

Group discussiion notes

  • Idea of having a student project negotiation
  • What is the purpose of the exam?
  • Balance between TMA and ECA/Exams
  • In theory low TMA grades could give low overall course grade but other way round is much more likely
  • Need to ensure that the TMAs are right to cover the course material
  • Comparision of projects to exams to check on the succe of the understanding of the course
  • Need exam questions that make students think rather than memorise
  • Often there is 2 months between TMAs - is this too much of a gap? Could more frequent iCMAs be the answer? Also can help tutos to check student progress
  • S104 has 9iCMAs 7 TMAs and and ECA - feedback from the students is positive
  • Issue is that if it is summative then students dont get immediate feedback
  • What about the possibility of have some form of substitution assignments where there is alot of assessment to help in terms of alleviating the pressure many students face in terms of keeping up with the course and doing regular study
  • Many subject areas use drip feeding to assist their increasing knowledge and undestanding of the course
  • At p/g level students are looking at themselves often and working with othes to acheive success
  • In the Business school use of self-assessment and finding materials online

Gráinne Conole
18:18 on 21 November 2009

Embedded Content

Assessment notes

Assessment notes

added by Gráinne Conole

Assessment presentation

Assessment presentation

added by Gráinne Conole

Plenary summary

Plenary summary

added by Gráinne Conole