SAT: What gives a tutor the X Factor: Examining the criteria for successful online learning facilitation (Sarah Quinnell)

Cloud created by:

Dr Simon Ball
3 February 2014

Establishing connection with students has long been acknowledged as an important factor in facilitating learning. Anyone involved in distance education will be familiar with the feelings of anxiety and isolation expressed by students studying away from campus. A close and successful face-to-face relationship with students is considered by many academics to be a powerful aspect of their professional identity, essential for creating meaningful learning experiences and in some cases, a contributing factor in their resistance to e-learning. This paper looks at how we create these personal relationships in an online environment. What does it mean to communicate vitality, conviction and feeling? What does ‘being there’ with students mean in an online learning environment? What do learners want from their tutors?

I identify and examine the factors and criteria that students use to rate the performance of their tutors and look at what it tells us, as e-learning practitioners, about what students value in their relationships with their tutors. Data was collected from four groups of students undertaking two vocational marketing certificates offered by UK organisations. Each group of students was asked to rate their Class Tutor on a scale of 1 – 5, where 1 is poor and 5 is excellent, after each class for a period of three months. At the end of this period the students were presented with a questionnaire that asked them to reflect upon the reasons for their ratings. A further set of focus groups, undertaken either in person or using synchrous messaging technology, were arranged to discuss the issues further. As both courses have face-to-face and online versions, which delivered the same materials, by the same tutors over the same period of time, I was able to compare criteria to begin to understand how technology impacts upon the relationships tutors have / can build with their students.

The research found that online learners place greater importance on what I classify as ‘performance’ criteria and how much they have been ‘entertained’ rather than the subject matter being presented. With reference to the impact of technology the data illustrated that in face-to-face teaching technological malfunctions had a negligible impact on tutor ratings, whereas for online students technical malfunctions cause significant negative impacts on tutor ratings. I go on to discuss the impact of these criteria on the implementation and further development of online learning environments, specifically in relation to setting and managing students experiences of online learning. The paper concludes by considering what this means for both learning technologists and tutors in the design and delivery of online learning content. While this paper begins to understand what learners value in terms of the tutors role in delivering online learning, in doing so, it raises a number of questions relating to how we look to meet these concerns and ensure that students expectations are managed and met prior to enrolment, during and after they have finished their qualifications. 

Extra content

Transcript slide 1: My presentation for H818 examines what the criteria are for successful online learning facilitation, if you like what gives an online tutor the X factor.

Transcript slide 2: The outline of my presentation is as follows: Firstly I will introduce the topic and give some background on why I chose it. Then I will briefly discuss the methodology used to collate empirical data. I will then discuss the key finding i.e. the performance factor. I then go on to briefly consider what this means for learning technology practitioners like ourselves. I will finish by talking about my future plans and how I am looking to develop this work further both practically and through Academic dissemination.

Transcript slide 3: My decision to examine this topic came from my practical experience of facilitating both face-to-face and online learning across a range of sectors. Recently I had the opportunity to support the same course delivered in two different ways. There was a completely online version and a completely offline version. In each version students were asked to rate their tutors out of 5 where 1 is poor and 5 is excellent at the end of every class. I began to notice marked differences in tutor ratings between the online and offline students. I found these variations interesting as it was the same tutor delivering the same information each time. In order for me, as the learning technologist, to be able to best support both the tutors and the students I wanted to gain an understanding of both how and why the online students viewed the tutors so differently. Basically what criteria were important to them or what gives an online tutor the X factor.

Transcript slide 4: So to do this I gathered both qualitative and quantitative data from students undertaking two vocational marketing courses. Both of which were delivered on and offline. After each class I asked the students to rank their tutors between 1 and 5 where 1 was poor and 5 was excellent. At the mid and end points of the course I distributed questionnaires that collated data through both closed and open questions in order to begin to develop an understanding not just of how they rated their tutors but also to begin to understand why and what criteria are important for online tutors to possess in order to be able to construct a pedagogy for / of online tutoring.

To date I have completed the initial analysis of these questionnaire. All 400 students participated in the initial tutor rating exercise. Of the 400 questionnaires distributed to students I have had 135 returned which gives me a roughly 33% response rate for this part of the study. Students were asked to identify themselves on both the questionnaire and the teacher rating exercise. This was done to enable me to compare scores with criteria and look at what criteria led to low or high scores and how they compared with the criteria important to their peers.

We have invited the 135 questionnaire respondents to join us at a series of face-to-face focus groups in order to gain further qualitative data looking at the reasons why students rate their tutors in certain ways. To date 48 respondents have agreed to attend the focus groups of these 36 are from the online course.

Transcript slide 5: From my review of the initial findings I can see that there are a number of marked differences between the criteria which give a face-to-face tutor the X factor and those which give an online tutor the same. Due to the time limitations for this presentation I will focus here on the criteria which matter the most to the online student. I have classified them as “performance” criteria as the overarching reason for giving higher ratings.

Performance criteria are largely classified as the way in which the content is delivered and the way in which it is received. I have broken those down into those the tutor is responsible for and those the learning technologist or IT support (I was both) are or could be argued are responsible for or at least responsible for supporting and designing.

The delivery criteria relate to:

Personality and Personalisation – This was the same for face-to-face students however, online learners see less of the normal social ques and conventions. Their vision and understanding of you as a tutor has to be delivered through a small box on a screen. If you are a face-to-face tutor your students are making judgements from the moment you enter the classroom. The environment that tutors frame themselves in online plays an important part in personalising the lecture and in making a connection as I will come to later.

Entertainment – Tutors who presented their content as part of a story or complex scenario drawing from real life experience gained higher ratings than those who just presented factual content. Initial findings suggest that once in front of a screen learners are looking to be entertained as well as educated. The rules which apply to good film and television production also apply in the presentation of learning content.

It’s not enough to just connect – Connecting with students has to be meaningful. For online learners successful tutors had to connect personally with the material they were delivering and find ways to connect the material to the learner. The use of student names and interactions were particularly important to learners.

Technology also played an important part in the delivery of the performance. The three criteria that I have highlighted here all relate to equality or equity of experience. Online learners want to be able to access their learning on any device, anywhere at anytime. In these courses it was particularly challenging as they were lecture style deliveries through the Adobe Connect platform (yes Blackboard Collaborate is better, if you were wondering). Obtaining equality of experience is difficult when you have multiple device types, app designs, internet access etc. A lot of plates to spin for the learning technologist, as it was in my case. The interesting thing however, was that if something went wrong the tutor got the blame and the rating they got reduced by 2-3 points. All the students knew me, they knew I was the techy one and so could easily have blamed me but a number of them have said it was not perceived to be my fault. The tutor should be in control of the technological experience.

I found these criteria quite surprising as I have never seen face-to-face tutors criticised because the projector or computer didn’t work in the room. It was normally tech support, if anyone who were blamed. I have never seen a lecturer blamed because they did not have control over the technology in a face-to-face class. Further questioning in this area illustrated why tutors were rated depending on their command of the technology – in a face-to-face class you can carry on in a different way. In an online environment if the technology fails, the whole experience fails. Online tutors therefore do live or die by the reliability of their technology.

Transcript slide 6: As I continue to analyse my data and complete the focus groups I am thinking about what impact or meaning this has for learning technologists or, indeed, anybody supporting teaching through online environments. I would be interested in hearing other people’s opinions in this area. For me going forward, just based on this initial data is that when designing online learning that we need to acknowledge to the learners that we understand the difference between learning through face-to-face teaching and learning online. Using this knowledge we should set and manage student expectation from the get go and be more pro-active about the shape and type of experience they will get. I also see a great deal of importance on being hand’s on to support both students and staff to ensure both groups can confidently navigate any online learning environment. The ability to do so confidently is key for a tutor to be rated as a successful tutor. The technology to me is key to the way in which tutors are perceived.

Transcript slide 7: So as we come to the end of my presentation where am I looking to go with this work. Well my future plans contain both a theoretical or academic component and a practical one. The data from this study will be written up and submitted to the Journal of Marketing and Higher Education for publication. Practically, My new job as Learning and Development Manager for the Macmillan Digital Education Group’s Mactrac project sees me training tutors in how to make the best use of online classroom spaces. I will be working with the Institute of Education to develop a pedagogy for online tutoring. I have really been able to link the work for this module into my real life as a learning technology practitioner.

 

 

 

Sarah-Louise Quinnell
14:22 on 15 February 2014

Embedded Content

Multi-media Poster

Multi-media Poster

added by Sarah-Louise Quinnell

What gives a tutor the X factor: Examining the criteria for successful online learning facilitation

What gives a tutor the X factor: Examining the criteria for successful online learning facilitation

added by Sarah-Louise Quinnell

Contribute

Laura Vella
9:45am 10 February 2014


Hi Sarah,  I'm not able to attend the online conference but would be really interested in hearing more about your work if there is another means of doing so.  Thanks. Laura

Debbie Grieve
3:04pm 10 February 2014


Hi Sarah, can't wait to attend. As a course designer and online teacher, I want to know if I am 'performing'! Thx :-)

Deb 

Jo Jacob
8:29pm 10 February 2014


It was lovely to see a preview of your slides at the weekend.  I don't think I will be able to attend on Saturday, so I will catch up with the recording later.  This seems highly relevant as so many of use are considering what the change from traditional classroon learning to online learning means for us as performers - and teaching is a performance after all.

Nicola Morris
9:22pm 12 February 2014


A useful output from this would be a form of questionnaire we could give to our students to solicit feedback. I too want to know if I'm delivering! Hope to catch your Talk but if not will be sure to hear the recording

Sarah-Louise Quinnell
9:59am 13 February 2014


Hi All, 

Thanks for the feedback to date - the presentation with transcript is currently in open studio. Nicola, I have a copy of the questionnaire I used when doing this if you are interested. 

Sian Lovegrove
2:34am 15 February 2014


I have cancelled my work today just to listen to your presentation! Well.... to be more accurate I have had my work cancelled so am now in the position to be able to fail to see more presentations! I am hoping yours will be visible to me as I have seen it grow in front of my eyes! Good luck doctor.

Sarah-Louise Quinnell
11:42am 15 February 2014


Thank You Sian, 

I want to see everyone from our tutor group if I can as they have all developed together :D

Dr Simon Ball
7:13pm 15 February 2014


Following the live presentations, we asked each speaker to respond to questions posed by audience members. In the short time available, it was not possible to put all of the questions submitted to the speaker for a response. We asked all speakers if they would respond to the unanswered questions here on Cloudworks. Here are all of the questions asked during the session:

  • Sarah - is the online tutoring synchronous?

  • Is there any difference if the tutors works with the same group in a blended programme. Ie you see them and then deliver other bits on line. Could one sway the other?

  • Is entertainment a commnet about the presenters (or 'tutors') or the material themselves...?

  • Does it depend on what the problem iwith the technology was? Some problems are often really the fault of the lecturer not being adept at using the technolgy, rather problems with the technology itself. maybe it would be interesting to see which problems were related to teachers and which to IT

  • Is it an issue that student expectations are often set by slick social networking sites

  • In managing student expectations might you include protocols for student involvement?

  • In your new role are you able or permitted to share you insights?

  • will you use your presentation to gain buy in in your new role

  • When or will 'academics' ever acknowledge their limitations as communicators and turn to a.n.other who can help with the way or how they present. My experience is that all, or most academics, are or wish to do it all themselves.

Sarah-Louise Quinnell
8:24pm 15 February 2014


So to answer the questions asked: 

1. The tutoring referred to in this presentation was synchronous. 

2. From the limited research data I have on blended scenarios the students involved saw each piece of delivery as a separate units. 

3. Entertainment relates to both the information and the way it is presented. 

4. The teaching took place in a live classroom like OU Live and the problems related to audio lag and lack of sync, connection problems etc which were out of the tutors control. 

5. I do feel that student expectations are set by slick social networking sites. You put a screen in front of people and they expect youtube. 

6. I think protocols for involvement would be very useful

7. I have shared my insights as part of my new role and will be hoping to build on what I have found.

8. I have already used the findings to gain buy in - it was a central part of my interview. 

9. The tutors I worked with certainly weren't do it yourself types BUT we were not working in traditional academic environments they were commercial / professional providers of education rather than HE where I do agree some people are scared of losing control. Sadly there is an inability to acknowledge that while you may be an awesome research you might not be the best communicator. Academics often do not want to admit they aren't the best at something. 

Cara Saul
10:26pm 15 February 2014


Really impressive to have this primary research Sarah

I was very interested by the way you teased some of the issues out about the different expectations of the online and face-to face learners. I was very shocked by your findings on how students recated to superficial factors like appearance. I wonder if this echoes some of the recent issues about social networking use. It may be that students expect slick well-presented presentations but perhaps they are more likely to objectify the tutor. I wonder if they feel more empathy in F2F encounters?

Sarah-Louise Quinnell
10:41pm 15 February 2014


Cara, 

Your presentation was fabulous, loved the way you put it together. 

Thanks for the comments, I was amazed by the superficial criteria that they used and the way they judged their tutors. I think the objectification of people in the media in combination with the presence of things like youtube and Ted talks that students have very different expectations of what they will be watching. 

I think there is more empathy in F2F encounters, I wonder if this is because they can establish a better and more genuine cognitive connection with the tutor?

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.