The web-site is now in readonly mode. Login and registration are disabled. (28 June 2019)

Cloud created by:

Alice (Xin) Huang
26 February 2013

I have just completed a one-year design and development process of an online e-learning course on e-learning - Applying e-learning opportunities. Ever since the first draft course map was done, I started at that time I didn't realise formative evaluation process. I consulted with subject matter expert and key stake holder at varies stage for finalised design plan, then  when the first module has been done, we used it as a prototype and carried out a more thorough consultation process and refind the module based on the feedback. Then the following 5 modules followed the same theme and principles. When all 6 modules have been done, we carried out a learnability testing (testing guideline shared by the link above)  and gathered more feedback from a learner's point of view, then finalised the whole course.  

To me the whole evaluation process I have carried out during last several month was a bit ad hoc, not planned carefully and purposefully. 

So, week 7 of OLDS MOOC appears to be just what I need – a systematic approach to evaluation. I really enjoyed the reading on  the formative evaluation paper,   and the opportunity to investigate different evaluation tools to evaluate online courses.

What I am going to do next is to develop a systematic approach for my work team,  for a new course or a program design, we need to  develop a set of tools and guidelines to make sure we collect useful data systematically that will help us to modify  courses  to enhance learning experiences.  Quote from Bob Ridge-Stearn’s  blog:

“Criteria for formative evaluation.

Reeves and Mor say there are no universal criteria established for formative evaluation of learning designs but that some of the most commonly considered are

  • functionality
  • usability
  • appeal
  • accessibility
  • effectiveness.

The inclusion of appeal, (“how much learners like it and why”) does not normally appear in the checklists I have been considering, but is an obvious question to ask and indeed we do include mid point and end point module evaluations in all our online courses. However, I must remember to look at these and see what they address as they were brought over from our face-to-face courses.

Collecting Data Systematically.

“The following formative evaluation methods are essential…”

  • peer review
  • expert review
  • learner review
  • usability testing
  • alpha, beta, field testing of prototype.”      

 

Extra content

Embedded Content

Contribute

Tom Reeves
3:03am 28 February 2013


Xin (Alice), I am so pleased that you see the potential of a more carefully planned approach to evaluation and how it might guide decision-making in the learning design work that you and your colleagues pursue. Formative evaluation is an especially good investment.  I have been part of a design team working on a very important new e-learning course for the World Health Organization. A beta-test version of the 11-week online course will be launched on Monday. There are dramatic differences between some of the earlier versions of the course design and where we are today with this course because of the rounds of formative evaluation we have done with this course throughout its development. I don’t have permission yet, but I hope I’ll someday be able to build a website that can illustrate for learning designers the before and after screen captures for how the course design was systematically enhanced over time on the basis of a range of formative evaluation strategies. The beta test that kicks off Monday is yet another type of formative evaluation.  The “learners” who will go through the beta version of the online course are all professionals who previously completed the traditional face-to-face version of this same course. Well, for now, thank you for your commitment to weave more and better formative evaluation into your learning design initiatives. – Tom Reeves

Alice (Xin) Huang
3:13am 1 March 2013


Thanks Tom for your comments.  I am very glad that I am doing this MOOC course which helped me to make sense a lot of things we were doing at work but not sure if we were on the right track. I like the idar to have a website to show people before and after, please let me know when you start building it, I would like to be involved and I am sure I will learn more from your experience. 

The e-learning course I mentioned just went live this Monday, we will continuously observing student's feedback, and incorporate their feedback to the next revision. 

Cheers,

Xin (Alice)

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.