MON: Designing for impact: rethinking video content for MOOCs (Allison Bell)

Cloud created by:

Allison Bell
6 January 2017

Massive Open Online Courses (MOOCs) can attract hundreds if not thousands of learners. As such, for course teams designing and delivering MOOCs, this can present many challenges. Learners can have many motivations and be diverse in their needs. Learner dropout rates can be high and producing MOOCs usually means significant investment in terms of budget and staff time. Making an impact and sustaining interest is therefore an important design challenge.

This presentation will give an overview of a project which is exploring implementation through a consideration of three key design challenges associated with the FutureLearn* model and designing for learning at a massive scale: (1) creating engaging video content, (2) creating optimal designs for learner interaction and collaboration and (3) designing quality automated feedback.

It will focus particularly on video. Video content can be quite costly to make; online course designs have to fight for attention; distractions are high; research tells us many students stop watching after a matter of minutes (Guo et al., 2014). So we consider: how can video make best impact? When is the use of video most effective?

We'll look at some initial findings in response to such issues, as part of a wider project to produce design guidelines. The project looks at research and existing guidelines offered by those active in this area, and also at MOOCs with higher than average learner retention rates. Difficulties associated with measuring impact in this area are also considered. Learner dropoff rates, and dropoff points, for example, offer a measure but cannot provide a complete picture (see Koller et al. 2013, and Ferguson and Clow 2015).

To view the project as the guidelines take shape, please visit:

https://sway.com/uRKXAbPxLn9erGYd

* FutureLearn is the UK's largest MOOC platform provider - visit www.futurelearn.com to view courses it offers.

References

Ferguson R. and Clow, D. (2015) ‘Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs)’, 5th International Learning Analytics and Knowledge Conference (LAK 15). Poughkeepsie, USA, 16-20 March.

FutureLearn (2016) ‘How does my course compare?’ FutureLearn Partners’ website [online]. Available at http://partners.futurelearn.com.

Guo, P., Kim, J. and Rubin, R. (2014) ‘How video production affects student engagement: an empirical study of MOOC videos’, Proceedings of the first ACM conference on Learning @ Scale Conference. Atlanta, 4-5 March, pp.41-50.

Koller, D., Ng, A. and Chen, Z. (2013) Retention and Intention in Massive Open Online Courses: In Depth (online), EduCause Review [online]. Available at http://er.educause.edu/articles/2013/6/retention-and-intention-in-massive-open-online-courses-in-depth.

Sharples, M. (2013) ‘Social learning and large-scale online learning’, FutureLearn, 11 September [blog]. Available at https://about.futurelearn.com/blog/massive-scale-social-learning/.

Extra content

Embedded Content

Designing for impact - session poster

Designing for impact - session poster

added by Allison Bell

Contribute

Dr Carol Waites
9:43pm 26 January 2017


This is a challenging area indeed.  I am running my own online course and find the qualitative assessment part time consuming.  Do you think users are prepared to pay for individual assessment?

I also remember in one FutureLearn course I reviewed that the instructors occasionally joined in the discussion forums even though it was a free course.  This seems to make it more popular and encourage interaction.  Is this supported in your view?

Allison Bell
10:04pm 26 January 2017


Thanks for your interest Carol.

Regarding paid for assessment, this is an area I haven't looked into in any detail but I think some users are prepared - I think for Coursera courses you need to pay for the assessment. So it comes down I think to your objectives behind the course in the first place.

Regarding the impact of instructor presence, yes there is research out there that indicates this has a positive effect - for example Dixson (2010) found that instructor-student communication clearly correlated with higher engagement in the course (186 survey respondents). The full article is 'Creating effective student engagement in online courses: what do students find engaging?'. I have personally observed the difference between visible and less visible instructor presence - being more positive and generating more comments with the former.

Leanne Johnstone
8:09pm 28 January 2017


Hi Allison,

I have enrolled on number of FutureLearn courses. Most of them I have dipped in and out of but I fully participated in the BLE course. I found the collaborative nature of the course really motivating, especially when the authors of the course commented and did a weekly round up of the conversations that had taken place.

I also really liked and valued some of the activities where I received and gave peer feedback. I dont think I would have paid for feedback unless I got some kind of accreditation. An interesting topic - I look forward to your presentation.

Best wishes,

Leanne

Mary Howell
3:20pm 29 January 2017


Hi Allison - I am looking forward to this presentation adn I am interested in your comments about the role of instructors/mentors adn this increasing engagement - that echoes what we are experiencing on the MOOC support team I am workig with.

Hi Leanne - perhaps I should interview you for my project?  What inspired you to choose and dip into the MOOCs whare that was your approach?  Do you think you drew lasting learning from doing that?  What was your motivation for fully completing the BLE one - how was that differnet to the ones you dipped into?  Don't worry you don't need to reply here...or t all ...

Stephen Gardiner
11:42am 1 February 2017


Hi Allison

I have done many of Futurelearn courses so this will be of interest to me.

Uffe Frandsen
5:21pm 1 February 2017


Hi Allison. As you might expect, I find this subject very interesting, especially the part on video engagement. Do you have any key findings to share in this regard? 

Allison Bell
8:38pm 1 February 2017


Hi Leanne/Mary/Stephen/Uffe,

Thanks for your recent comments. I have decided, due to time constraints, that I'll only be able to go into detail on one of the challenges - and that will be video (so yes, Uffe, there will be some key findings!). Mary - I am happy to share thoughts around the role of mentors/hosts outside of this - it will also be included in my guidelines that are currently a work in progress in Sway (about to put the link into the abstract!). Leanne - yes I think I'd agree that paying for feedback without accreditation would be a hard sell..

Allison

James Fanning
11:33am 4 February 2017


Allison, I wondered if you had any view in relation to where MOOC design sits in compulsory education, e.g. secondary school level. Are there a whole set of digtial skills that really should be a core part of teacher skills - especially in realtion to the design of digital content - or (and I come from a secondary school background) is this something we leave to others (whoever the others may be)?

Heather Bloodworth
2:04pm 5 February 2017


Hi Allison, as you know I feel your pain in relation to the challenges with MOOC!   I think focusing on the video content/engagement  is a good idea. In my experiance, videos within an elearning course does break up the monotony and reduces the reading load. However, poor video qualit can be distracting and sometimes even counderproductive.  Looking forward to the presenbtation and your findings.  

Allison Bell
8:48pm 5 February 2017


Thanks for your comments James and Heather.

James - I think digital skills should be a part of teacher training (I'm afraid I know little about teacher training - but would have assumed it would be included); so many aspects of digital life are essential so if we think it's important for students to develop these skills then it's essential educators have them - not least from a safeguarding point of view (and maybe Jude could have more to say here?!).

Heather - I completely agree re: variety and the counterproductive effect of poorly thought through design/execution - something I'll talk about in my presentation :)

 

Paul Curran
1:48pm 8 February 2017


THis is something I am quite interested in. I have been experimenting with Adobe Edge Animate which allows for the creation of interactive videos, amongst other things. One of the things I've picked up from social media is the value of interaction even if it is just clicking to proceed.

Good luck!

Allison Bell
8:19pm 8 February 2017


Hi Paul - thanks for your comment. I haven't heard of Adobe Edge Animate so will have to check it out. Yes I must admit I didn't go down that avenue too far, I think because I've encountered too many 'click, click and here's a basic memory quiz', type approach, but still I could do to keep an open mind and investigate further for other examples of this kind. If you have seen any good examples, please let me know! Allison

Dr Simon Ball
10:49am 14 February 2017


Hi Allison
Please find below the main questions and comments from your live presentation. It's up to you how to answer them, whether you wish to group them, or whether you wish to point to an answer already given above, for example.
Best wishes
Simon

  • How do they define 'dropout'? Could some only be registering to grab materials and have no intention to stay?
  • Could students be getting the information they need and then leaving?
  • How does the drop out rate compare to other ‘free courses’?
  • I am also interested in the definition of dropout. I've seen at least one study that looked at 29 MOOCs and found a *completion* rate of 6.8% (Parr, 2013), so a dropout rate of 7% to 15% is hard to match up with that?
  • Do you think that, with so much video material available now, we are becoming more accepting of less glitzy production?
  • ... and is dropout a problem? If I buy a boook to find out about a topic, have I not 'done it right' if I don't read the whole book?
  • Is glitzy a msrepresentation of the lecturing that actually happens in f2f at that uni?
  • Is there evidence about whether MOOC participation coverts to other forms of participation?
  • I enjoyed your paper Alison. I LOATHE videos in moocs, and now I know why!
  • Not really related, but has any research been done about why individuals don't complete MOOCs?

Allison Bell
1:14pm 14 February 2017


Thanks everyone for your comments during the session.

I think dropouts are to be expected given the diversity of learners with different motives for registering for the course - good reasons for why learners don't complete, for example, are offered by Daphne Koller (co-founder of Coursera) and colleagues here: http://er.educause.edu/articles/2013/6/retention-and-intention-in-massive-open-online-courses-in-depth

What counts as a 'dropout' can be interpreted in different ways - for example learners who then unregistered themselves, learners who were 'dormant' - who did not progress, learners who did not complete (again different measures can be used - complete all the activities and/or complete the assessment).

But my contention is that learners could be retained for longer if better choices were made around how the learning is designed and presumably if you have clearly defined your 'target learners' - if you are designing for them - then the learning should be as well-matched as possible to their needs.

The question around production values, e.g. quick webcam take versus studio-produced, is one I haven't found conclusive evidence on. Both Guo (2014) and Hibbert (2014) looked at this and what their research has in common is that the personalisation - the human touch that the educator was engaging with you as a learner directly - was valued highly. (Link to Guo is in above abstract; link to Hibbert is here: http://er.educause.edu/articles/2014/4/what-makes-an-online-instructional-video-compelling.)

Regarding whether MOOCs convert to other forms of participation - if this is getting at whether it helps convert learners onto accredited programmes then I have seen evidence of this but I don't have more supporting evidence other than what I've personally seen / heard about. I think it's a natural ambition for educational institutions though in terms of student recruitment.

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.