ICT and Success - Conclusions from the ARC Meta-synthesis
The Association pour la recherche au collégial (ARC), whose mission is to promote and support research in colleges, maintains a bilingual inventory of publications on ICT in the journal Pédagogie collégiale. Last year, some 55 empirical research projects were selected from the inventory to conduct a meta-synthesis of their results. A meta-synthesis offers more than just a simple summary. It follows a qualitative approach that analyzes the data and research findings to identify co-occurrences between relevant variables.
ARC has already produced a meta-synthesis from thirty research projects on the impact of ICT in college teaching. A thematic dossier on this study appeared in English in Pédagogie collégiale in 2011. The updating of the research inventory increased the number of studies to a total of 55. Making a new metasynthesis of this enlarged corpus was one of ARC's goals for 2012.
The project coordinator for this was Sean Gayadeen. His systematic approach was supervised by Christian Barrette and revealed some interesting observations on 30 years of ICT use.
ICT Results According to Targeted Goals
The experiments were divided into three categories according to their ultimate goal:
- experiments to improve students' grades
- experiments to increase deep conceptual learning (complex cognitive operations)
- experiments to support and develop academic motivation
Only 40% of experiments that focused on improving educational outcomes achieved positive effects. In nearly 10% of these experiments there was actually a negative effect. Even though it seems it is difficult to improve students' grades using ICT, there were positive results in 40% of these cases. The analytical goal in this category was to identify the conditions that generated the success of these experiments.
The picture was very different with research aimed at improving deep conceptual learning among students. In fact, 80% of experiments in this area reported positive results. Apparently, skills for generalization, discrimination, transfer, complex problem solving, critical thinking and metacognition benefit from educational activities using ICT.
For academic motivation, 66% of the research projects had a positive impact on the students. Here as well, identifying the conditions for success of these experiments was the goal of the study.
Why has ICT had so little effect on students' grades while enhancing complex cognitive operations related to deep conceptual learning? There are several possible explanations.
It is possible, that evaluation methods that do not take "significant" learning into account are one reason for this situation. Educational activities that engage students in reflection, critical thinking, hypothesis formulation and debate sometimes are assessed using evaluations of rote learning, repetition and routine. If this is true, the mixed record of technological change in education could be the result of a mixed record of assessment practices that do not fit the context of deep conceptual learning.
Winning Conditions for ICT Integration
Analyses of data from the 55 research projects used by ARC for the meta-synthesis provide particularly interesting observations when their variables are combined to check what the distinctive conditions for successful experiments are.
Included among the winning conditions are:
- Good Planning
Good planning always characterized the activities that had a positive impact on grades, motivation and deep learning. Whether proper planning implies a strong consistency between learning modes and methods of assessment remains to be seen.
- Consistency between the pedagogical approach of the teacher and the technical devices operated:
- With a pedagogical approach focusing on transmission of content by rote (performance approach), the most useful devices are presentation or demonstration software, virtual exercises and training tutorials
- With an educational approach that supports students in their mastery of essential content, the most useful devices are virtual tutorials or simulators for the exploration of complex realities. These technologies were useful insofar as they provided feedback to guide students in their own acquisition of knowledge
- With a social constructivist pedagogical approach for the co-construction of knowledge and collaborative work, students benefitted from communication platforms and virtual team work.
The data collected by the meta-synthesis contains much information that remains to be analyzed and understood. It is already possible, however, to observe conditions consistent with the three desired effects. The figure below summarizes this information.
Positive or Negative Conditions Affecting Research Outcomes
A meta-synthesis generates strong trends from the analysis of co-occurrences of variables taken into account in a research protocol, however, they are also at work in daily activities, developed every day by hundreds of actors in the network. As teachers, you are among those experimenters on the front lines! Do you have any comments to share? Do the observations reported here confirm your findings, or do they contradict them?