The web-site is now in readonly mode. Login and registration are disabled. (28 June 2019)

Liz Masterman: From proof of concept to evidence of impact: towards a principled approach to evaluating learning design tools

Cloud created by:

Yishay Mor
5 October 2011

The increasing maturity of learning design tools means that it is no longer sufficient to evaluate them merely according to the criterion of 'proof of concept'. Rather, we now need to demonstrate their effectiveness in helping teachers to improve their design practice and, ultimately, students' learning. This paper draws on the experience of designing the evaluation of the Learning Designer tool in order to map out the issues in developing a theory-informed evaluation framework. It begins with a brief description of the project and its objective to have an impact on teachers' design practice, and then explores the principles that have guided the formulation of the evaluation framework in three areas: i) our conceptual understanding of 'impact', ii) a qualitative approach to measuring impact and iii) the changes in practice for which we might seek evidence. It then describes the early implementation of the framework in formative evaluations of the Learning Designer and indicates the methodological and logistical barriers to be overcome. The paper concludes that a principled approach to evaluation is integral to the science of learning design.

Extra content

Embedded Content

added by Yishay Mor

From proof of concept to evidence of impact (slides)

From proof of concept to evidence of impact (slides)

Accessible Alternative
added by Liz Masterman


Adi Kidron
1:51pm 13 October 2011

1st Discussion group:

  • asses effectiveness of LD tools
  • how can you evalute a tool when theres not enough work carried out yet?
  • the need to evaluate the tool within the context of its implementation environment.
  • evidence of impact - what are they? how do you measure?

Adi Kidron
1:53pm 13 October 2011

  • a "potenial for an impact" - combination of the tool itself and the way you're using it.

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.