This is an installment in a series of summaries of journal articles that I have been reading.
Van Leeuwen, A., Janssen, J., Erkens, G., & Brekelmans, M. (2014). Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL. Computer & Education, 79, 28–39.
Computer-supported collaborative learning (CSCL) generates a lot of data through student interaction. Instructors must sift through and make sense of this data before they can give appropriate guidance. Previous research looked at the types of student interaction and instructor guidance, but there is a gap in the literature in how instructor guidance may be influenced by different presentations of collaboration data. The purpose of this study is to examine the effect of participation analysis tools that on instructor diagnosis and guidance.
The sample consisted of 28 high school teachers and student teachers. 14 teachers were randomly assigned to the control group (no supporting tools) and 14 were assigned to the experimental group (supporting tools). Each teacher was given 4 vignettes which consisted of authentic collaborative situations. Each vignette consisted of several collaborative groups and each group varied in collaborative and cognitive aspects. Three types of data were collected: instructor actions during the vignettes, instructor interventions (messages sent to students), and overall diagnosis of participation and discussion for each group.
The results indicated that the frequency of interventions, focus of the interventions, and receivers of the interventions differed between the control and experimental groups. The authors concluded that the statistical supporting tool helped instructors identify problems with participation. Additionally they found that instructors using the support tools judged non-problematic groups less harshly. One result, the lack of difference in focus on collaborative aspects between the experimental and control groups, surprised the authors.
The study had two key limitations: lack of data for one of the supportive tools and variables that could have affected the instructors’ focus on collaborative versus cognitive aspects of the student interaction. The general implication of this study is the conclusion that supporting tools can affect teachers’ guidance of groups.
This study was an interesting experiment in CSCL because it looked at an intervention to encourage positive interaction among students, but it focused more on the tool that the ways that an instructor should use it. The authors hypothesized that the support tools would help instructors better diagnose problems with participation and discussion, but they results did not confirm this hypothesis. This may have been influenced in part by the lack of distinction between cognitive and collaborative aspects within the design of the study. The authors do not describe how the tools support these two different aspects of participation; they mention only that the tools support participation in general. The authors do not clearly explain whether they are trying to foster cognitive or collaborative participation.
An unaddressed aspect of this study was the temporality of the interaction. Each vignette played in 8 minutes, and the instructor sent messages during that time. This suggests that the results of this study are limited to single session synchronous collaborative learning. Future research should examine these learning tools with asynchronous group work when the interaction is taking place over days or weeks. The actions and behavior in synchronous and asynchronous CSCL may need to be interpreted differently by instructors because the delays in responses to messages and the frequency of messages may have different causes in the two types of situations.
Many of the instructors in this study were student teachers. It is reasonable to think that a veteran teacher may be more adept at identifying interaction patterns and problem groups without the support tools. The authors should analyze their data from this perspective to see if experience affects the level of problem diagnosis.