SAT: What gives a tutor the X Factor: Examining the criteria for successful online learning facilitation (Sarah Quinnell)
Cloud created by:
Dr Simon Ball
3 February 2014
Establishing connection with students has long been acknowledged as an important factor in facilitating learning. Anyone involved in distance education will be familiar with the feelings of anxiety and isolation expressed by students studying away from campus. A close and successful face-to-face relationship with students is considered by many academics to be a powerful aspect of their professional identity, essential for creating meaningful learning experiences and in some cases, a contributing factor in their resistance to e-learning. This paper looks at how we create these personal relationships in an online environment. What does it mean to communicate vitality, conviction and feeling? What does ‘being there’ with students mean in an online learning environment? What do learners want from their tutors?
I identify and examine the factors and criteria that students use to rate the performance of their tutors and look at what it tells us, as e-learning practitioners, about what students value in their relationships with their tutors. Data was collected from four groups of students undertaking two vocational marketing certificates offered by UK organisations. Each group of students was asked to rate their Class Tutor on a scale of 1 – 5, where 1 is poor and 5 is excellent, after each class for a period of three months. At the end of this period the students were presented with a questionnaire that asked them to reflect upon the reasons for their ratings. A further set of focus groups, undertaken either in person or using synchrous messaging technology, were arranged to discuss the issues further. As both courses have face-to-face and online versions, which delivered the same materials, by the same tutors over the same period of time, I was able to compare criteria to begin to understand how technology impacts upon the relationships tutors have / can build with their students.
The research found that online learners place greater importance on what I classify as ‘performance’ criteria and how much they have been ‘entertained’ rather than the subject matter being presented. With reference to the impact of technology the data illustrated that in face-to-face teaching technological malfunctions had a negligible impact on tutor ratings, whereas for online students technical malfunctions cause significant negative impacts on tutor ratings. I go on to discuss the impact of these criteria on the implementation and further development of online learning environments, specifically in relation to setting and managing students experiences of online learning. The paper concludes by considering what this means for both learning technologists and tutors in the design and delivery of online learning content. While this paper begins to understand what learners value in terms of the tutors role in delivering online learning, in doing so, it raises a number of questions relating to how we look to meet these concerns and ensure that students expectations are managed and met prior to enrolment, during and after they have finished their qualifications.
Transcript slide 1: My presentation for H818 examines what the criteria are for successful online learning facilitation, if you like what gives an online tutor the X factor.
Transcript slide 2: The outline of my presentation is as follows: Firstly I will introduce the topic and give some background on why I chose it. Then I will briefly discuss the methodology used to collate empirical data. I will then discuss the key finding i.e. the performance factor. I then go on to briefly consider what this means for learning technology practitioners like ourselves. I will finish by talking about my future plans and how I am looking to develop this work further both practically and through Academic dissemination.
Transcript slide 3: My decision to examine this topic came from my practical experience of facilitating both face-to-face and online learning across a range of sectors. Recently I had the opportunity to support the same course delivered in two different ways. There was a completely online version and a completely offline version. In each version students were asked to rate their tutors out of 5 where 1 is poor and 5 is excellent at the end of every class. I began to notice marked differences in tutor ratings between the online and offline students. I found these variations interesting as it was the same tutor delivering the same information each time. In order for me, as the learning technologist, to be able to best support both the tutors and the students I wanted to gain an understanding of both how and why the online students viewed the tutors so differently. Basically what criteria were important to them or what gives an online tutor the X factor.
Transcript slide 4: So to do this I gathered both qualitative and quantitative data from students undertaking two vocational marketing courses. Both of which were delivered on and offline. After each class I asked the students to rank their tutors between 1 and 5 where 1 was poor and 5 was excellent. At the mid and end points of the course I distributed questionnaires that collated data through both closed and open questions in order to begin to develop an understanding not just of how they rated their tutors but also to begin to understand why and what criteria are important for online tutors to possess in order to be able to construct a pedagogy for / of online tutoring.
To date I have completed the initial analysis of these questionnaire. All 400 students participated in the initial tutor rating exercise. Of the 400 questionnaires distributed to students I have had 135 returned which gives me a roughly 33% response rate for this part of the study. Students were asked to identify themselves on both the questionnaire and the teacher rating exercise. This was done to enable me to compare scores with criteria and look at what criteria led to low or high scores and how they compared with the criteria important to their peers.
We have invited the 135 questionnaire respondents to join us at a series of face-to-face focus groups in order to gain further qualitative data looking at the reasons why students rate their tutors in certain ways. To date 48 respondents have agreed to attend the focus groups of these 36 are from the online course.
Transcript slide 5: From my review of the initial findings I can see that there are a number of marked differences between the criteria which give a face-to-face tutor the X factor and those which give an online tutor the same. Due to the time limitations for this presentation I will focus here on the criteria which matter the most to the online student. I have classified them as “performance” criteria as the overarching reason for giving higher ratings.
Performance criteria are largely classified as the way in which the content is delivered and the way in which it is received. I have broken those down into those the tutor is responsible for and those the learning technologist or IT support (I was both) are or could be argued are responsible for or at least responsible for supporting and designing.
The delivery criteria relate to:
Personality and Personalisation – This was the same for face-to-face students however, online learners see less of the normal social ques and conventions. Their vision and understanding of you as a tutor has to be delivered through a small box on a screen. If you are a face-to-face tutor your students are making judgements from the moment you enter the classroom. The environment that tutors frame themselves in online plays an important part in personalising the lecture and in making a connection as I will come to later.
Entertainment – Tutors who presented their content as part of a story or complex scenario drawing from real life experience gained higher ratings than those who just presented factual content. Initial findings suggest that once in front of a screen learners are looking to be entertained as well as educated. The rules which apply to good film and television production also apply in the presentation of learning content.
It’s not enough to just connect – Connecting with students has to be meaningful. For online learners successful tutors had to connect personally with the material they were delivering and find ways to connect the material to the learner. The use of student names and interactions were particularly important to learners.
Technology also played an important part in the delivery of the performance. The three criteria that I have highlighted here all relate to equality or equity of experience. Online learners want to be able to access their learning on any device, anywhere at anytime. In these courses it was particularly challenging as they were lecture style deliveries through the Adobe Connect platform (yes Blackboard Collaborate is better, if you were wondering). Obtaining equality of experience is difficult when you have multiple device types, app designs, internet access etc. A lot of plates to spin for the learning technologist, as it was in my case. The interesting thing however, was that if something went wrong the tutor got the blame and the rating they got reduced by 2-3 points. All the students knew me, they knew I was the techy one and so could easily have blamed me but a number of them have said it was not perceived to be my fault. The tutor should be in control of the technological experience.
I found these criteria quite surprising as I have never seen face-to-face tutors criticised because the projector or computer didn’t work in the room. It was normally tech support, if anyone who were blamed. I have never seen a lecturer blamed because they did not have control over the technology in a face-to-face class. Further questioning in this area illustrated why tutors were rated depending on their command of the technology – in a face-to-face class you can carry on in a different way. In an online environment if the technology fails, the whole experience fails. Online tutors therefore do live or die by the reliability of their technology.
Transcript slide 6: As I continue to analyse my data and complete the focus groups I am thinking about what impact or meaning this has for learning technologists or, indeed, anybody supporting teaching through online environments. I would be interested in hearing other people’s opinions in this area. For me going forward, just based on this initial data is that when designing online learning that we need to acknowledge to the learners that we understand the difference between learning through face-to-face teaching and learning online. Using this knowledge we should set and manage student expectation from the get go and be more pro-active about the shape and type of experience they will get. I also see a great deal of importance on being hand’s on to support both students and staff to ensure both groups can confidently navigate any online learning environment. The ability to do so confidently is key for a tutor to be rated as a successful tutor. The technology to me is key to the way in which tutors are perceived.
Transcript slide 7: So as we come to the end of my presentation where am I looking to go with this work. Well my future plans contain both a theoretical or academic component and a practical one. The data from this study will be written up and submitted to the Journal of Marketing and Higher Education for publication. Practically, My new job as Learning and Development Manager for the Macmillan Digital Education Group’s Mactrac project sees me training tutors in how to make the best use of online classroom spaces. I will be working with the Institute of Education to develop a pedagogy for online tutoring. I have really been able to link the work for this module into my real life as a learning technology practitioner.
14:22 on 15 February 2014