Week7 Xin (Alice)'s evaluation experience
Cloud created by:
Alice （Xin） Huang
26 February 2013
I have just completed a one-year design and development process of an online e-learning course on e-learning - Applying e-learning opportunities. Ever since the first draft course map was done, I started at that time I didn't realise formative evaluation process. I consulted with subject matter expert and key stake holder at varies stage for finalised design plan, then when the first module has been done, we used it as a prototype and carried out a more thorough consultation process and refind the module based on the feedback. Then the following 5 modules followed the same theme and principles. When all 6 modules have been done, we carried out a learnability testing (testing guideline shared by the link above) and gathered more feedback from a learner's point of view, then finalised the whole course.
To me the whole evaluation process I have carried out during last several month was a bit ad hoc, not planned carefully and purposefully.
So, week 7 of OLDS MOOC appears to be just what I need – a systematic approach to evaluation. I really enjoyed the reading on the formative evaluation paper, and the opportunity to investigate different evaluation tools to evaluate online courses.
What I am going to do next is to develop a systematic approach for my work team, for a new course or a program design, we need to develop a set of tools and guidelines to make sure we collect useful data systematically that will help us to modify courses to enhance learning experiences. Quote from Bob Ridge-Stearn’s blog:
“Criteria for formative evaluation.
Reeves and Mor say there are no universal criteria established for formative evaluation of learning designs but that some of the most commonly considered are
The inclusion of appeal, (“how much learners like it and why”) does not normally appear in the checklists I have been considering, but is an obvious question to ask and indeed we do include mid point and end point module evaluations in all our online courses. However, I must remember to look at these and see what they address as they were brought over from our face-to-face courses.
Collecting Data Systematically.
“The following formative evaluation methods are essential…”
- peer review
- expert review
- learner review
- usability testing
- alpha, beta, field testing of prototype.”