Abstract: A reflective account is presented of experiences which took place alongside a research project and caused a change in approach to be more sceptical about implementation of learning technology. A critical evaluation is given of a previous e-assessment research project, undertaken from a position of naive enthusiasm for learning technology. Experiences of teaching classes and designing assessment tasks lead to doubts regarding the extent to which the previous project encouraged deep learning and contributed to graduate skills development. Investigations of the benefits of another technology—in-class response systems—lead to revelations about learning technology: its enthusiastic introduction in isolation cannot be expected to produce educational benefit; instead it must address some pedagogic need and should be evaluated against this. Overall, these experiences contribute to a shift away from a naive enthusiasm to an approach based on careful consideration of educational need before technology implementation.