The web-site is now in readonly mode. Login and registration are disabled. (28 June 2019)

If you don’t know where you are going…how do you know if you have arrived?

Presentation by Adrian Kirkwood at CALRG 2012 conference.

Cloud created by:

Rebecca Ferguson
19 June 2012

Several years ago, Margaret Cox and Gail Marshall (2007) argued that many questions remained unanswered regarding the effects of ICT upon students’ learning in school contexts. Having identified many weaknesses in the approaches adopted in research studies conducted over numerous decades, they asked “Do we know what we should know?” and identified “the need for a thorough, rigorous, and multifaceted approach to analysing the impact of ICT on students’ learning” (p.60). They were critical of the instruments and methods used in many studies, and also questioned some of the underlying assumptions made about learning, effects of different teachers’ pedagogical approaches and the particular types of ICT use. A recent review of research literature relating to the use of technology for teaching and learning in higher education (Price and Kirkwood, 2011) identified many similar concerns.

The proposed session will explore some of the methodological issues associated with research and evaluation studies that attempt to demonstrate enhancement of student learning or the student experience in higher education through the use of technology. Although much depends upon what teachers/researchers deem to be important (Kirkwood and Price, 2012), so often the nature of ‘learning’ is not made explicit: instead it is (erroneously) taken for granted that there is a common, shared understanding among teachers and among students. The session will draw attention to the limitations of various research approaches and question the usefulness or suitability of certain means of evidence collection that are commonly used. Above all, it will argue that greater clarity is required in specifying not only the desired or expected outcomes from any ‘technology in education’ study, but also the evidence deemed appropriate to demonstrate that those outcomes have been achieved.


Extra content

Embedded Content


Rebecca Ferguson
9:57am 19 June 2012

Kirkpatrick's four-level model of learning evaluation is summarised on Wikipedia.

The four levels are:

  1. Reaction - what participants thought and felt
  2. Learning - the resulting increase in knowledge and/or skills, and change in attitudes
  3. Behaviuor - transfer of knowledge, skills, and/or attitudes from classroom to the job
  4. Results - the final results that occurred because of attendance and participation

Contribute to the discussion

Please log in to post a comment. Register here if you haven't signed up yet.