Liz Masterman: From proof of concept to evidence of impact: towards a principled approach to evaluating learning design tools
Cloud created by:
5 October 2011
The increasing maturity of learning design tools means that it is no longer sufficient to evaluate them merely according to the criterion of 'proof of concept'. Rather, we now need to demonstrate their effectiveness in helping teachers to improve their design practice and, ultimately, students' learning. This paper draws on the experience of designing the evaluation of the Learning Designer tool in order to map out the issues in developing a theory-informed evaluation framework. It begins with a brief description of the project and its objective to have an impact on teachers' design practice, and then explores the principles that have guided the formulation of the evaluation framework in three areas: i) our conceptual understanding of 'impact', ii) a qualitative approach to measuring impact and iii) the changes in practice for which we might seek evidence. It then describes the early implementation of the framework in formative evaluations of the Learning Designer and indicates the methodological and logistical barriers to be overcome. The paper concludes that a principled approach to evaluation is integral to the science of learning design.