WED: E-feedback - supporting teachers in their use of Turnitin and GradeMark (Lisa Hale)

Cloud created by:

Lisa Hale
11 January 2016

At my current higher education institution, most student coursework is submitted and marked as a hard copy and findings of a recent in-house survey show that many students do not collect feedback on their summative assessed work. Why is this and what can we do as educators to change this?

Despite the importance of feedback to learning (Gibbs and Simpson, 2004), learners and tutors complain about feedback. Learners say that they cannot access their feedback easily enough or the feedback process takes too long, and then they are often unable to read their tutor’s handwriting, or feel the feedback should be more personalised and specific. The result? They do not collect it. Tutors say that their learners do not engage in the feedback process and complain that their learners do not collect their feedback from them. Tutors also say that it takes a significant amount of time to produce good written feedback.

Implementing an e-feedback policy could answer some of these issues. Learners can access their feedback at their convenience, read typed comments more easily, benefit from improved feedback through in-text annotations, comments and popup clarifications and advice which can give more specific and directed feedback, and also benefit from the use of audio feedback. Consequently, learners are more likely to access, read, and use their feedback to feed forward to future assignments.  For tutors, an e-feedback policy is more efficient without the need to manage a considerable amount of paper. In addition, tutors can benefit from being able to provide quicker and improved feedback through the use of in-text comments and audio tools, which could engage learners more in their feedback and encourage further dialogue about their learning.

There are positive findings in the literature regarding the use of an innovative tool in Turnitin – GradeMark (Buckley and Cowap, 2013; Chew and Price, 2010). Turnitin is a software programme which helps learners and tutors to prevent plagiarism and supports best practices for the use and citation of other people’s material (Turnitin, 2010). GradeMark enables a tutor to mark work online (through Turnitin) and easily return feedback and marks to their learners. Institutions who are using this software and tool say that learners are more likely to read and reexamine their feedback due to its convenience.

In order to support tutors working with international students on a pre-sessional English for Academic Purposes course in their implementation of e-feedback, I am developing a series of specific support resources (for example How to guides). These will help me to provide consistent support to these tutors due to the high turnover of staff to meet the demands of, in particular, summer semester courses. Moreover, as these resources will be stored in an open access repository, tutors can return to them when needed.  

In my multimedia conference presentation, I will briefly outline some of the issues surrounding feedback and how the implementation of e-feedback could address these concerns. Then I will show the repository that I have developed and some of the resources that I am creating. 

Extra content

Embedded Content

Project artefact

Project artefact

added by Lisa Hale

Contribute

John Baglow
6:42pm 16 January 2016


Lisa, this is a topic very close to my heart (and presentation) I work in F.E. but the ITE courses I teach on probably share some similarities to H.E. teaching.

My presentation will look, (as part of a wider exploration of collaboration between learners and dialogue between tutor and learners),  at how the feedback process can be made more effective.And I too am basing my project on a teacher-training module, rather like you. That does raise some issues I haven't thought through yet e.g. in the conference do I deliver it as though the audience are trainee teachers (least likely option) or do I deliver it as information as a description of what I shall do with the trainees? How are you doing it?

I sometimes give my students videocast feedback. Some like it, others prefer written feedback. Maybe I will give them the choice. Did you catch the recent research (I don't know where I saw it!) which concluded that in a university somewhere students were asked how they liked to get feedback and then there was some sort of trial which apparently concluded that the students improved more when receiving their least favoured form of feedback? 

John Kerr
12:12pm 18 January 2016


Hi lisa,

Like John I too am very intersted in this topic.I have used Turnitin for a number of years and I have mix views regarding it.

Anyway, some helpful feedback hopfully:
Why a policy? My experience shows that policies are difficult to implement and with feedback can stiffle any other forms of innovations, i.e. video feedback. Perhaps a framework may be better suited?

is there any consideration given to external markers reviewing this feedback? I know you can download the PDF verison of comments but the audio remained on the site, at lest it use to. Perhaps this is too granular for this presentation.
Finally, have you looked at any research that shows that electronic feedback is received / picked up / acted upon by students anymore than handwritten feedback? I understand the argument of the anytime/anywhere access but does that make students act upon it? Perhaps a qucik search could back this up.

Looking forward to this.
John  

Anna Orridge
3:47pm 19 January 2016


Hi Lisa,

This is an interesting topic for me, as my experience is mainly with pre-sessional students too, and my project aims to support EAP teachers. For that reason, I am think I may well follow your work for the EMA. Would that be OK?

I think that the tutors will appreciate your How-To guides, and this will avoid you having to repeat instructions. If later 'versions' of TurnItIn appear, do you think it will be reasonably easy to adjust the resources?

As regards the difficulty with getting students to read and respond to their feedback, have your tutors considered giving short face-to-face feedback sessions alongside the written comments? I have found that students are far more likely to think carefully about how to adjust their next assignments if this is put in place.

 

Kiran Gawali
5:08pm 19 January 2016


Hi Lisa, Looking forward to this presentation! I would like to follow you as I'm involved with staff training and Turnitin could be utilised better. We are using grade book feedback in some of our programmes , some staff and students are benefitting from it. But a lot of staff have resisted because they find it more useful to annotate the actual assignment.

Lisa Hale
5:51pm 20 January 2016


Hi John (B)

Thanks for your comments.

I'm planning on delivering my presentation as if I were at a conference which could have quite a wide range of participants rather than to the teachers I would actually be using my training materials with. So I'm going to have to be careful about using terminology that those attending my session may not know about or have much experience of. 

I'd be interested to know why some of your students prefer written feedback - do you mean in an actual hard copy format or online? I like the idea of giving students a choice about which type of feedback they would prefer - however, does this add to your workload with having to do different types of feedback and do students change their minds which can get a bit confusing?

I'd be very interested to read the feedback article you are referring to - sounds fascinating :)

Lisa 

John Baglow
10:04am 21 January 2016


Lisa, because of my current obsession with ways of making the feedback process more effective I am constantly asking my ITE students what they prefer and how they react to different modes of feedback.

Reasons for face-to-face oral feedback: more personal, opportunity for a dialogue, tutor can be more nuanced in making a point, tutor can react to how the student is responding But: one-to-one feedback is very time-consuming

Feedback using podcasts or videocasts: less opportunity for real dialogue, but has most of the other advantages of oral feedback. In addition, there is a record of it. One student said it was more laborious to go back to a videocast than to revisit written feedback

Written feedback: online feedback can be very immediate (no handing in of hard copies) and can be referred to more speedily. Also, the tutor can modify it in the light of work subsequently submitted eg I may feed back to a student that I would like to read more about how they motivate their learners. I say this in the feedback and when they submit it I can comment on that immediately.

Online marking and feedback is a time and effort-saver for the tutor. It gives the External Examiner very easy access to student work.

I shall hunt for the article about student choices and feedback.

Lisa Hale
3:09pm 22 January 2016


Hi John (K)

Thanks for your comments - some interesting points.

I agree that a policy can imply that there are restrictions and as John B mentioned in his comments - perhaps students could be asked which type of feedback they would prefer you to give on a particlar piece of work. The type of feedback that my project is looking at supporting teachers with is as a result of students submitting work through Turnitin - so many tutors are not using this plagiarism software and students are submitting their work in a hard/paper version. There would be nothing to stop a tutor using video or audio feedback - which Turnitin does offer but what my institution is looking to stop are piles of hard copies of summative assessments and tutor feedback left uncollected. 

As we use Turnitin via our virtual learning environment, we give our external markers access to our Moodle sites so they can see samples of work. 

For the second part of TMA02 - the extended abstract - I looked at quite a bit of research on e-feedback and even the use of GradeMark, and it was all very positive. However, much of the research was qualitative into how students and staff felt about using the new tech and tools (compared to what they normally used/experienced) and I thought there was a lot of the 'halo' effect - so users were excited about using the new tools and 'everything was great'. Then there was other research from data collected on VLEs in terms of students accessing their online feedback - but this didn't really look at how long they accessed it for - just that they did access it - and then the study tried to make a correlation into different types of students and whether this affects why they do or don't access feedback. But there was virtually no research into how students are using this online feedback/how they are acting on it - are they using it differently? I also couldn't find anything on a link between online feedback and student attainment - so does using online feedback improve students results? This is another area I'm interested in as the National Student Survey results for assessment and feedback are always much lower than other areas and I wonder if it's not because the students don't understand or access feedback, but they don't know how to act upon it and when they do, they can't see what difference it makes to their results.

Do you use much video and audio feedback? I use audio quite a lot on formative work but I haven't really used video feedback - can you recommend a tool?

I'm sorry John - I realised I've waffled on a bit in this reply!

Lisa

 

Lisa Hale
3:57pm 22 January 2016


Hi Anna

Thanks for your comments and I'd be delighted if you chose to follow my work. I'm starting to put together my presentation and some of my project resources and I'll put them on OpenStudio. 

It's a good point about updating resources and I think it would be relatively easy as I'm planning on limiting the length of my 'How to...' guides to about 3 or 4 minutes - so it wouldn't take me too long to redo those. I really want to break it down into small sections for two reasons - firstly, so that teachers can go off and quickly practice that particular point and secondly, they may only need to know about one small point and they don't want to watch a long tutorial just to find that out. 

I agree that it's vey difficult to beat the short synchronous (F2F or online) tutorial to look at formative feedback and get the students engaged in a dialogue. However, I feel there is also an issue with summative assignments and feedback and this is where online feedback might help. I think it's about helping students to access and understand the importance of feedback not just for the modules and assignments that they are doing at that point, but also future modules and assignments. 

Lisa

Lisa Hale
4:07pm 22 January 2016


Hi Kiran

Thanks for your comments and, again, I'd be delighted if you chose to follow me. 

Your point about staff being resistant as they say it's more useful to annotate the actual assignment is an interesting one. Do you mean the actual paper assignment? I wonder why they say it's more useful.

There has also been some resistance in my institution to the use of GradeMark as staff say that it takes longer to do the online annotations. I don't disagree with this but I do think it's a question of practice and getting used to it. I create my own Quickmarks - annotations with comments - as I find the generic ones difficult for my international students to understand and once you get into the swing of things, I find it quicker and easier to use. One of the problems I have is getting a general overview of an assignment - so I would previously flick backwards and forwards through an assignment and you get a good overview of it - this is more difficult to do using this online system. The other issue I have noted is with a draft system - so the international students on my programme do drafts of assignments and we compare drafts to see what changes have been made and what feedback has been acted upon - this is also more challenging to do using this system. Finally, students HAVE to double or triple space their work - otherwise using the in-text annotations is a NIGHTMARE.

Lisa

John Kerr
11:16am 26 January 2016


Hi Lisa,

 

I am doing some research on student attainment being linked to the delivery of feedback. Nothing to share as of yet but we should have a paper out for April on it. If you check the Glasgow University website, you should see something around that time appear - perhaps via the Enlighten Database.

NSS and feedback - now that's a can of worms! NSS scores on feedback have always been a problem area for institutions. Feedback takes place throughout the year and includes Tutorials etc. but do students perceive that as feedback? Perhaps not. Again, can of worms :)

Anna Orridge
3:54pm 26 January 2016


Hi Lisa,

I've had trouble with in-text citations too, so it's good to know that it's just me. I'm a short story writer, and publishers and readers often want to send back drafts with comments, but the number of times I'm unable to access them is just perplexing. There must be a simple solution to this, because writing between the lines is clumsy and makes it hard for people to understand what has been written.

Lisa Hale
12:30pm 30 January 2016


Hi John (K)

I'll look out for your research - sounds very interesting :)

Chris Gray
3:52pm 30 January 2016


Lisa

Your project acts as a nice complement to John's (B.). I do think your project can be of great assistance to students. Having feedback that is readily accessible should help them to be a more reflective learner and should improve their performance.

Will you be looking at ways of speedily disseminating student feedback, possibly considering including some form of fun social emoticon, or similar, to act as a means of engaging students in their feedback which would enable them, if they wished, to Tweet or Facebook it, to share with friends.

Is there a way of readily collating student feedback for each individual so they have an easy to hand progression overview as well? I find students are usually very receptive to constructive feedback, but they soon forget a lot of suggestions.

Elizabeth Frost
6:08pm 31 January 2016


I tried to use audio for feedback for some A Level English Lang and Lit students, and they hated it.  I thought I would be clever and ask the students to email me their work; I would record my feedback and email back to them so they could access the marked work in their own time.  One of the main reasons why this was so unsuccessful was because they were unable to vocally justify (argue) their choices with me. 

Is Turnitin expensive software to purchase?  I haven't had any real experience with this, other than being shown how the plagarism element works, but I understand it is quite popular.

Do you think that students don't see the feedback because they have been distracted by a physical mark? 

I'm not sure I would use video feedback due to the fact that I would be concerned that students would find some way to 'doctor' this and post on Facebook, bearing in mind that I teach in FE and HE students are probably more mature?

John Baglow
11:21am 1 February 2016


Lisa, I use screencast-o-matic for screencasts. I don't know if it is any better than alternatives but I have persevered with it and got the hang of it. One of the potential issues with anything like this is that sometimes other people can't access your masterpieces. I wouldn't claim to have cracked that issue with SoM but I find that it gives me various options for saving my clips. I can save them to SoM itself, to YouTube (both of these options then give you a unique URL for that clip) or to my own PC as an mp4 file. It is useful to have these options I find. 

SoM is free but I pay a massive $15 per year for the Pro version - to be honest, I can't even remember why I went for that. One amazing thing about SoM is that if you email them with a query you get a personal reply very quickly.

Here is a link to an extract from one of my clips:

http://screencast-o-matic.com/watch/colebChTCb

Lisa Hale
11:36am 4 February 2016


Hi Chris

Thanks for your comments. Yes, I've chosen to follow John B as our topics have the 'feedback' connection :)

Unfortunately, the GradeMark tool within the Turnitin software is quite an inflexible system in that you can't add 'gadgets' or 'widgets' - I'm really not sure if this is what they are called - which would allow you to instantly share your feedback via, for example, Facebook or Twitter. However, one of the reasons I'm creating these more specific 'How to...guides' is that it will help teachers create their own set of feedback comments (known as Quickmarks) rather than use the generic one produced by the software/tool. This way teachers will be able to add their own style/personality to their feedback comments and, hopefully, create more personalised feedback which will engage their learners.

Your point about collating student feedback so that students can see overall progress and quickly flick from one piece of feedback to another - is an interesting one. I hadn't thought about this before. Turnitin allows you to set up multiple drop boxes for assignments - so you can have a 'first draft' and 'final draft', for example. So students can jump between tutor feedback between drafts of an assignment - but they can't jump between feedback from different assignments - this is a drawback of the system. Perhaps this is where a learning portfolio (or something like a Padlet board) would be useful - students could download and keep all their feedback in one place (Turnitin does allow you to download feedback) - which would be more accessible. I like that idea - I'll see if I can think about how to do that :)

Lisa

Lisa Hale
12:16pm 4 February 2016


Hi Elizabeth

Thanks for your comments. Were your students recording an audio response to your audio feedback? It sounds like a really interesting idea - what a pity they didn't like it. Have you thought about having another go and if you did, would you do it in a different way? I've only ever used audio feedback myself to give feedback and not asked students to use it. I don't use audio feedback either - I know some tutors are big fans - they say their students say the feedback feels more personal, less formal and more like a dialogue. 

I admit to knowing absolutely nothing about the price of Turnitin as it is something I have access to via my institution and it is available on all our course VLEs. Sorry :(

Your point about students being distracted by a mark is an interesting one - we all do it, don't we - look at our mark first - well I do! It's also the first thing you see when you get into the OU online TMA/EMA service - you then have to download your assignment folder and open up the marked assignment file before you can see your feedback. Is there an argument that students who get a 'good' mark or who are happy with the mark they see, don't access their feedback? And the student who gets a 'bad' or 'fail' mark also doesn't access their feedback as they feel 'what's the point?' or 'it's all going to be negative feedback and I don't want to see that' - or perhaps they feel afraid of seeing it?  Have you ever tried withholding a formative mark until a student has acknowledged they have read their formative feedback and produced an action plan or short piece of reflective writing on the feedback? A colleague of mine tried it last semester with very positive results :)

Lisa

Lisa Hale
12:20pm 4 February 2016


Hi John (B)

Thanks for the Screencast o matic recommendation and tips - I've been impressed with the quality of the screencasts that both you and Nicki have produced using this tool - I think I'm going to give it a go for my first 'How to... guide' for my project and see how I get on.

Lisa

Elizabeth Frost
9:47pm 8 February 2016


Hi Lisa

 Is there an argument that students who get a 'good' mark or who are happy with the mark they see, don't access their feedback? Yes, I think there is.  The students think that they have fulfilled the criteria and don't look any further.  And the student who gets a 'bad' or 'fail' mark also doesn't access their feedback as they feel 'what's the point?' or 'it's all going to be negative feedback and I don't want to see that' - or perhaps they feel afraid of seeing it?  I agree to all of your statements above. That's why it is so important to make sure the feedback is constructive and non-judgemental.   Have you ever tried withholding a formative mark until a student has acknowledged they have read their formative feedback and produced an action plan or short piece of reflective writing on the feedback? A colleague of mine tried it last semester with very positive results :) That is a really good idea.  I'm going to give that a try with the mock exams we have coming up soon.  I have asked all my maths staff to just give a narrative with no formal mark.  It would improve the students maths skills to count the mark up themselves anyway!

Lesley Hamilton
10:45pm 8 February 2016


Hi Lisa,

We have been using all 3 of the Turnitin applications: Originality, PeerMark and GradeMark for over 5 years now. Adoption has been gradual but the instuitional culture is such that Turnitin is now taken for granted, i.e. the accepted norm. However, a lot of staff to this day are using Turnitin without having engaged with the numerous staff development opportunities that have been set-up for them: including bespoke screencasts on it's set-up and use.Their learning has been informal via via colleagues or having to contact me when issues arise.

The statistics showing the number of times our guides and video tutorials hae been accessed is very low compared to the number of staff we have. How do you intend to engage staff? To get them to view your Guides?

Last year we were able to get some funding that allowed us to evaluate the best options for staff wanting to mark online using GradeMark. We evaluated marking:

  • using a desktop computer with two monitors - staff were very positive about this option. It allowed them to mark and have an overview of the whole paper, e.g. marking in one window whilst having the ability to scroll through the submission within a second window.
  • Using the iPad app - staff liked the option to mark offline then synchronise when back in a wifi area. The option gave staff a greater sense of flexibility and choice over where they marked. Interestingly, some staff have commented that they now use the iPad app as a second monitor - alongside a desktop or laptop.
  • Using a Laptop - feedback was neither positive or negative. Although having a laptop like using the iPad app allowed staff greater flexibility over where they marked.

I agree with your comment regarding QuickMark sets. The system QuickMarks are 'American' and are not as meaninful. I think, that with all software we learn so much and then we use what we already know. So when I  staff what they can do with QuickMarks they are then begin to see the potential for reducing the time it takes to mark a submission and also how they can improved upon consistency of marking - especially if you have several markers marking 400 submissions. A shared QuickMark set is invaluable.

You haven't mentioned the Rubric manager. Do you have any advice regarding how to engage staff and improve the development of rubrics? Again, I'm thinking in terms of improving consistency of marking.

Sorry, I have appeared to have rambled. Turnitin takes up bit of my time - so when I read your abstract... I look forward to seeing your presentation.

 

 

 

 

 

Maxine Armstrong
5:35pm 10 February 2016


Hi Lisa,

My university uses Turnitin and GradeMark for the submission and marking of student assignments. I would be interested in hearing how successful you have been to implement this e-feedback policy.

You talk about students being more likely to use e-feedback and tutors finding it more efficient, just wondering how much evidence you have for these views and have you found any counter-arguments?

I see you are implementing Turnitin and GradeMark through Moodle; do you have any integration problems? We use Blackboard and there are sometimes gremlins.

Have you used the rubrics feature in GradeMark? If so, was it beneficial? There has been some limited interest in using this feature at my university, but I don’t think it has been adopted much.

Do you find any accessibility issues with using e-feedback, either as text or audio?

I am looking forward to your presentation on Wednesday and hearing more about your open access repository for resources.

Best wishes, Maxine 

Maxine Armstrong
5:40pm 10 February 2016


Hi Lesley,

I have just read your comment and note your use of the iPad app. When this was introduced it was quickly adopted by some staff, but then when it was investigated by the IT people they found it very insecure, so now our policy does not recommend it. It has been a constant bugbear with some having to be online to mark assignments.

Best wishes, Max

Kiran Gawali
6:41pm 11 February 2016


Hi Lisa, You asked me if staff preferred making annotations on paper, answer is yes. The 'resistance' towards using grade mark derives from some things mentioned by others here, syncing issues using the iPad, no option for blind double marking, technical apprehension, quick marks have been rather useless. Having said all this , originality checker is used across the institution. Majority of staff have been ok with that. Very few want these originality reports printed( sigh). Inevitably we have admin staff doing triple amount of work because grademark is not implemented. So yes if you implement a policy and back up with robust staff training sessions you will stand a better chance then we have. Hope to learn some strategies from you! :)

Dr Susan Morris
10:47am 16 February 2016


Hi Lisa, 

This is an interesting topic for my tutoring work in HE.  In OU Science we use Turnitin and the use of colour is important in showing the intensity of plagiarism from each source.  Feedback using scenarios would be an interesting intervention.  I remember well a case where an international student had no history of plagiarising in their studies and then in their final module in the final assessment plagiarised 90% of their work from 9 sources.   It was so frustrating and so revealing that an intervention earlier in their degree pathway would have prevented an epic fail for their module and investigations of whether Turnitin had missed earlier... false negatives?

Best wishes, Susan 

Lisa Hale
3:09pm 17 February 2016


Hi Lesley

Thanks for your comments. It sounds like we've had similar experiences with the QuickMarks and adoption by teachers. Only last week an experienced colleague asked me if I could show her how to link her assignment rubric to her Turnitin assignment. I casually asked her if she'd been to the in-house training which ran last year - she said 'no' as she hadn't had time and had heard that it was too general. Interesting!

The results of your evaluation of marking using GradeMark are interesting. I would agree with the flexibility that using an iPad or laptop offers. Also your finding about using a second screen - or iPad/laptop as a second screen. I think this point highlights one of the slight disadvantages of the system - it can be difficult to get a feel for the whole piece of writing and it's not so easy to flick back and forwards through a piece of writing - so you could do that more easily on a second screen. Many of the written assignments on our pre-sessional courses have a draft system - so again using 2 screens would make it easier to go between the drafts. 

I'm planning on introducing my resources to new teachers during their induction and then following up just before they would be required to use Turnitin/GradeMark - I'll see how this goes with the summer teachers this year. 

The pre-sessional programme that I work on is modular so each module has a module leader who has the responsibility of setting up the Turnitin assignment dropbox and attaching the standardised assignment rubric to that assignment - so there's no pressure on the tutor to be involved in this stage unless they were interested in learning how to do so. We've only just started using the rubric manager this year - and I'm a big fan already - as a module leader myself - it's very easy to attach rubric to an assignment - and as a teacher using the rubric to mark the student's work. Other tutors have been positive about it too (they say it's quick and easy to use and means that all the students receive the same experience) - and there is still a general comments area for the tutor to make some personalised comments. One of the 'how to guides' that I want to create will be about how to mark a student's work using the rubric attached to the assignment.

I note that you mention PeerMark - we don't use this yet - I don't think we have access to this tab even - I'd be interested to hear how you find it.

Lisa

Lisa Hale
4:06pm 17 February 2016


Hi Maxine

Thanks for your questions - I'll try to answer them one-by-one

I would be interested in hearing how successful you have been to implement this e-feedback policy.

My HE institution has university-wide projects which are part of a programme to enhance the overall student experience and online marking and assessment is one of eight current projects - it started in January 2015 and I think they've just completed the literature review stage and networking with other institutions who are already doing this. At the end of last year, there were lots of generic workshops to support staff with their use of Turnitin/GradeMark and in particular the rubric manager. So I'd say that across the University it's still in the early stages with pockets of advanced use or 'champions' as we call them. I'd say that our department was an early adopter - I've been using Turnitin and GradeMark for about 4 years and all the tutors are using it at some point in their teaching. 

You talk about students being more likely to use e-feedback and tutors finding it more efficient, just wondering how much evidence you have for these views and have you found any counter-arguments?

Yes, there is quite a lot of evidence to support these views - the 2 studies I mention in my short abstract - Buckley and Cowap & Chew and Price - both these studies report on the findings of tutors - which was useful for my project and theme. JISC also has also done a lot of work on this area and has guides and case studies in support of the electronic management of assessment. Finally, there was an interesting literature review by Hepplestone et al. 2011 (I referred to it today in my presentation and the full reference can be found in the version of my ppt under the links tab above). Yes, there are also counter arguments - these centred around technology issues, training and support issues, and some accessibility issues. One of the other problems I had with the literature was that there was very little research on the exact nature of the student engagement with feedback - so many studies just meaure IF the feedback is accessed - perhaps not even how long for, or how students are or could be using the feedback. I'd also like to know more about what impact, if any, the use of online feedback has on student learning - John K in this thread mentions some research that he's doing in this area - so I'm going to keep an eye out for that :)

I see you are implementing Turnitin and GradeMark through Moodle; do you have any integration problems? We use Blackboard and there are sometimes gremlins.

Personally, I've had very few. It comes up as an optional activity that we can add to our Moodle sites and this then takes us through to the Turnitin site where we set up the assignment details. Occasionally, there have been issues but I think these are more from Turnitin and we get emails when there are issues there or server issues which may affect it. A few years ago, GradeMark would sometimes freeze and you had to be really careful as if you closed the tab or tried to refresh the screen, you could lose some of your work - but again I haven't really experienced those issues so much now.

Have you used the rubrics feature in GradeMark? If so, was it beneficial? There has been some limited interest in using this feature at my university, but I don’t think it has been adopted much.

I'm a big fan of the rubrics feature. I've been using it for about a year now. Once you have set up your rubrics template, you can just 'attach' it to an assignment - so it means that all the tutors marking are using the same rubrics/marksheet - but you can still make personalised comments using the comments and annotations and overall general comments. The tutors who I've been using it with have also been positive about it - they only have to click on the grade/number for a particular criterion. You can also programme the system to calculate the mark or grade. We input the mark ourselves.

Do you find any accessibility issues with using e-feedback, either as text or audio?

I haven't as yet. Before the students access the system for the first time, the tutors do a one-to-one tutorial with them. We show them how to enlarge their work on the screen, print off their work if they would prefer not to read it on the screen, access the comments in a consecutive manner - as sometimes looking at the marked script can be a little overwhelming - and access their audio feedback. At the moment, I have a student who is partially sighted so I'll be interested to work with him to see whether he has any issues and what adjustments we can make to support him.

Lisa

Lisa Hale
4:08pm 17 February 2016


Hi again Maxine

I also note in your post to Lesley about the use of iPads not being recommended due to security issues - this is an concern and a real pity. I'm going to follow it up with our IT department too.

Thanks

Lisa 

Lisa Hale
4:17pm 17 February 2016


Hi Kiran

I think one of the key things you mention is the QuickMarks - the generic ones, as Lesley says, are American and completely useless. I feel that we overcame a big hurdle in terms of getting people on board when we asked module leaders to take responsibility for creating and sharing specific sets of QuickMarks for their international students and a particular assignment. I've now had requests from tutors to colour code my QuickMarks and connect them to the different rubrics in the marksheets. 

I note your point about blind double marking - this would be an issue. Luckily most of our coursework assignments submitted electronically are part of a draft system so our 'anonymous marking' policy doesn't apply as you've worked too closely with students for it to be truely anonymous. But we do do sample blind moderation and we do have to print off a top, middle and bottom sample to do this - so not so good :(

Lisa

Lisa Hale
4:25pm 17 February 2016


Hi Susan

Thanks for your comments.

What an interesting case! I wonder whether there were other factors which influenced this this student. It seems so strange that there hadn't been evidence of plagiarism in their other work - I wonder if there had but the tutor hadn't spotted it - just a thought - you always think that you'll be able to spot it! In addition, Turntin sometimes doesn't pick up everything. I've been convinced that something was cut and pasted from a source but it didn't show up on Turnitin. By putting it into Google - I found it!

We use Turnitin formatively - so we use it early on in the course for a draft of the final assignment. This means that the students see it as a way of helping them rather than punishing them. It also means that we pick up on those who have 'cut and pasted' or done 'patch plagiarism' and we can then use their work in a formative way to show them how to change it for their final draft. 

Lisa

Dr Simon Ball
5:53pm 17 February 2016


Hi Lisa

Here is a summary of the questions/comments from your presentation - please respond as you wish:

  • I think the 'personalisation' thing is key. I wonder how we can help students feel that what they are reading is specific to them.
  • Yes, interestingly, we did a bit of informal research on feedback and speed of return was almost more important than quality. It became 'irrelevant' as far as students were concerned, once they'd moved on to the next bit of the course. Seeing it as feedforward wasn't in much evidence.
  • Note John Baglow's work on video and audio feedback
  • Teachers resent the fact that they end up writing so much on students' papers and they don't get much in the way of a response.
  • I think that spoken feedback always comes across better. The problem is, it's very time consuming - and what does everybody else do while you are delivering it?
  • Verbal feedback can be recorded.
  • I wonder whether the task of giving prompt feedback could be improved by staggering assignment due dates, instead of everyone handing in at the same time. Would that help or just make it more complicated?
  • That's a really interesting idea. I suppose the 'fairness' issue could get in the way. Students would wonder why their peers were getting more time.
  • Did I miss it or are they free programmes?
  • I like the way the student would be able to find the annotation so quickly and easily.
  • W are discussing gertting feedback out first then marks
  • Possible to get student's to read their feedback first and to use this to try to estimate their grade? Once they have estimated their grade, they get their actual grade?
  • Is marking anonymous?
  • the student self-assessment is SO revealing, particularly about how many students under-estimate their achievement (even at the v top end!)

Lisa Hale
12:39pm 18 February 2016


Hello everyone

Thanks for all your comments and questions - I'll try to answer them one-by-one below each comment/question.

  • I think the 'personalisation' thing is key. I wonder how we can help students feel that what they are reading is specific to them.

Yes, I agree - anonymous marking and personalised feedback. I think we need to address the student directly in our feedback and make reference to a specific part of their assignment, where possible - unless it's an overall comment - and follow this up with a suggestion or recommendation to feed forward. For example 'Lisa, this is an interesting point but could do with an example to support what you are saying - could you refer to your own experience here?' (I've just made this example up, by the way).

  • Yes, interestingly, we did a bit of informal research on feedback and speed of return was almost more important than quality. It became 'irrelevant' as far as students were concerned, once they'd moved on to the next bit of the course. Seeing it as feedforward wasn't in much evidence.


This is a very interesting finding and I would suspect that if we asked something similar with our students, they would say the same. We try to turn around our formative marking within a week - but I still get students asking me before that if I've marked their work. Because I teach f-2-f, I don't like to return work one-by-one as I mark them; I like everyone to have it back at the same time. I am able to see if students are using my feedback - and their peers - in their formative work as we have multiple formative assignments before the summative ones. However, it really does depend on the student - some have admitted that they don't look at the feedback at all - just the grade/mark, whereas others I can see go through their work with a highlighter and seek clarification and want to engage with you about it. I've started to get my sts to bring their formative assignments and feedback to the next class and we do an activity with it so I can show them how to feed forward. Or I ask sts to write a short reflective statement/action plan about what they are going to take from one assignment to another. 

  • Note John Baglow's work on video and audio feedback
  • Verbal feedback can be recorded.

I've put these 2 comments together. Yes, I'm following John and have picked up a few ideas - especially asking sts about what type of feedback they would prefer. I have a st this semester who is partially sighted and I've asked him and he'd like me to do screencast feedback on his written work - so I'm going to try that. I often do audio feedback on presentations and discussions - and have then asked students to record a response to my feedback - that's worked well. 

  • Teachers resent the fact that they end up writing so much on students' papers and they don't get much in the way of a response.


Yes, when I asked my colleagues informally, many of them mentioned this point. I think that some students just don't know what to do with the feedback and are unsure how to respond, so that side of it could be scaffolded with activities. Also, perhaps the teacher needs to start the dialogue - I know they already have by providing the feedback but by noting another means of responding, so the option of office hours/tutorial/online tutorial to discuss or a follow up action plan/piece of reflective writing. Since studying with the OU, it's been interesting to note how they deal with summative work - we get an overall feedback response but work is not returned - whereas our TMAs are returned and we get intext comments and overall ones. 

  • I think that spoken feedback always comes across better. The problem is, it's very time consuming - and what does everybody else do while you are delivering it?


I agree - there is alot of scope with spoken feedback - whether f-2-f or online - but yes, it can be time-consuming - how long do you allow? 5, 10, 30 mins?? Is it the same for all sts? I probably wouldn't do f-2-f feedback with everyone - I'd hold 1-1 tutorials or small group tutorials if it was groupwork - and I'd probably do this out of class time in office hours - but I know not everyone has the luxury of being allocated this time in their workload. I think that recording spoken feedback would solve the issue of what everyone else does while you are doing it - but again this would be have to be done outside of class time. Tutorials and f-2-f/online feedback is another area, as Peter spoke about in his presentation - on some courses they are not well attended or those that do attend are often not the ones that need as much support. As I work with international sts, we make our tutorials a compulsory part of their overall attendance on their pre-sessional courses - so they are well attended :)

  • I wonder whether the task of giving prompt feedback could be improved by staggering assignment due dates, instead of everyone handing in at the same time. Would that help or just make it more complicated?


This is an interesting idea and you'd like to think that some thought had been given to assessment schedules and workload for tutors and students when assignment dates were set but ....... I think this decision needs to be taken at a programme level and, yes, it would help tutors to return feedback more promptly and students to stagger their assessment workload. I think it is complicated though, as you point out, as there seems to be a pattern to continuous assessment dates - for example after blocks of 4 weeks or midway through a semester/course and, of course, all the summative dates. 

  • That's a really interesting idea. I suppose the 'fairness' issue could get in the way. Students would wonder why their peers were getting more time.


I think this comment might be in response to the previous one? My understanding of the previous comment is that different assessments on different modules would be staggered not different deadlines for students doing the same assessment on the same module. Yes, you're right this would be completely unfair and how would you choose who would hand in their assignment on which date?

  • Did I miss it or are they free programmes?


No it's not free. I have access to the programme through my institution's VLE. I read this article from the THE - there have been some complaints about how expensive it is. It's very difficult to get an idea of the cost - on the Turnitin website there is just a 'contact one of our sales reps' tab. I must admit I don't know what other institutions use - it would be interesting to hear about alternatives. John K earlier in this discussion thread mentioned that he had mixed views about it. Currently, I'm happy with the way we use it - we use it formatively so students upload drafts to it and then we look at the results and how they could use what it shows them - so when they upload their summative assignment to it, it's not the first time with possibly very negative and serious consequences if there is a high match result.  

  • I like the way the student would be able to find the annotation so quickly and easily.


Yes, students really like this feature and if tutors have created their own set of QuickMarks, it's even better. Students can either look out for the speech bubbles or QuickMarks on their actual uploaded work, or there is the option to view these annotations page-by-page as a sort of list - you click on the list and it takes you through to where it is on the paper. So depending on how a student prefers to work, they can see their comments either way. Some of my students prefer this second method as they said that looking at work that has quite a large number of comment bubbles or annotations can be a bit overwhelming. And of course, if they don't want to view it on the screen, they can download a pdf with everything on it and work through the paper version. 

  • W are discussing gertting feedback out first then marks
  • Possible to get student's to read their feedback first and to use this to try to estimate their grade? Once they have estimated their grade, they get their actual grade?
  • the student self-assessment is SO revealing, particularly about how many students under-estimate their achievement (even at the v top end!)


I'll comment on these 3 together as there is a connection. A colleague on another programme had very positive results from doing something similar to this - withholding grades/marks until students have responded to their feedback in the form of an action plan. They have also not given any marks for an early formative assignment - only feedback - interesting! Another colleague has asked their students to grade their own assignments and been surprised at the low marks they give themselves (there's always one or two students who give themselves A+), and then seen the reaction from the students when the tutor reveals the marks that they had given them, which are usually higher. I don't think they have done it quite as you suggest here, which is using the tutor feedback to estimate the grade given - I'll pass this on to them and I might try this myself :). Yes, I agree even at the top end, there is that self-doubt, lack of certainty about whether they've answered the question correctly or missed something out - I don't think there is a huge issue with underestimating your mark but for example if a student thinks they've failed or only just passed and you've given them an A, then perhaps there are other issues that need to be addressed here. I work with international students (coming from different educational backgrounds) and many of them feel uncomfortable about assigning themselves a grade that might 'disagree' with what their tutor thinks. Others have said to me that they don't think it's their 'job' to give it a mark - interesting!

  • Is marking anonymous?


You have the choice to make the assignment anonymous or not. You have to decide this when you are setting up the assignment dropbox and you can't go back on that decision. We have an anonymous marking policy - however, on my programme we have a lot of draft assignments - first and final - so these are exempt from the policy as we've worked too closely with the students on their first draft for it to be possible to mark anonymously. We do blind moderation (or sample second marking) thoug, but we have to print these off as this is an issue with Turnitin - there is no feature to allow second/blind marking - as the original marker's comments, etc. are visible. We've tried to get around this several ways - once we chose the samples of work to be blind marked before everyone started marking and we did those first and noted comments/marks privately and then discussed them and then the first marker put their comments, etc on the actual paper. But the feedback from tutors was not very positive about this - they felt it held them up a little as they wanted to 'get on with their marking' rather than wait for the standardisation to take place - so we went back to printing off of sample scripts. I'm not sure how others have dealt with this?

Thanks for all the though-provoking comments and questions.

Lisa

 

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.