If you don’t know where you are going…how do you know if you have arrived?

Several years ago, Margaret Cox and Gail Marshall (2007) argued that many questions remained unanswered regarding the effects of ICT upon students’ learning in school contexts. Having identified many weaknesses in the approaches adopted in research studies conducted over numerous decades, they asked “Do we know what we should know?” and identified “the need for a thorough, rigorous, and multifaceted approach to analysing the impact of ICT on students’ learning” (p.60). They were critical of the instruments and methods used in many studies, and also questioned some of the underlying assumptions made about learning, effects of different teachers’ pedagogical approaches and the particular types of ICT use. A recent review of research literature relating to the use of technology for teaching and learning in higher education (Price and Kirkwood, 2011) identified many similar concerns.

The proposed session will explore some of the methodological issues associated with research and evaluation studies that attempt to demonstrate enhancement of student learning or the student experience in higher education through the use of technology. Although much depends upon what teachers/researchers deem to be important (Kirkwood and Price, 2012), so often the nature of ‘learning’ is not made explicit: instead it is (erroneously) taken for granted that there is a common, shared understanding among teachers and among students. The session will draw attention to the limitations of various research approaches and question the usefulness or suitability of certain means of evidence collection that are commonly used. Above all, it will argue that greater clarity is required in specifying not only the desired or expected outcomes from any ‘technology in education’ study, but also the evidence deemed appropriate to demonstrate that those outcomes have been achieved.