Sequential Analysis of Online Group Interactions

This study explored and developed tools to examine patterns in student interactions in online threaded discussions, focusing on interactions that support critical thinking and argumentation (e.g. state position, state agreement or disagreement, provide supporting arguments, evaluate arguments, define evaluation criteria, etc.). Use of the tool enabled this study to address the following questions: When a argument or claim is followed by a critical response versus a counter-argument, which interaction is most likely to elicit greater discussion in subsequent threads? After each of these two interactions, what types of messages are most likely to follow? Likewise, what happens when a position statement is followed by an supporting argument versus a position statement followed by opposing arguments? What kinds of critical thinking can be expected to immediately follow each type of interaction (3-event sequences) and beyond? What is the average length of discussion threads that follow each interaction? When and how often do students follow-up Arguments with Evaluation and critical assessment of their arguments? Which interactions occur with probabilities statistically and significantly lower or higher than expected probabilities, or are significantly different from probabilities prescribed by existing models of critical thinking or rules of argumentation?

To examine these processes in group discussions, the Discussion Analysis Tool (DAT) was developed to conduct a quantitative analysis of event sequences using the methods of sequential analysis (Bakeman & Gottman, 1995). DAT was developed in Visual Basic using Microsoft Excel to support content analysis and measurement of event sequences and their transitional probabilities in threaded discussions - functions that are computationally intensive and infeasible without the appropriate tools. The findings of this study showed that DAT was very effective in identifying patterns in student interactions and measuring the events that follow specific interactions. For example, the probability of sustaining a discussion thread was highest when the discussion is initiated by the exchange of opposing ideas, viewpoints or disagreement. Presented arguments were rarely followed by evaluation of the arguments to test their significance, validity and accuracy. Instead, evaluations were most likely to be raised during negotiation of group concensus or drawing of conclusions. Disagreements were rarely stated explicitly in discussions. Qualitative analysis showed that disgreements were more likely to be stated implicitly through the presentation of opposing arguments and critiques.

The tools and methods developed in this study provided the means to obtain a birds-eye view of the complex processes and patterns of interaction that occur in threaded discussions. DAT also provided the means to examine group interactions, processes and associated outcomes using both quantitative measures (transitional probabilities) and qualitative/graphical descriptions (state transitional diagrams). The implications for future research are the following: 1) identify and measure unique patterns in group interactions and the outcomes that follow; 2) empirically test and identify dysfunctional as well as constructive forms of interactions; 3) empirically test the effectiveness of different instructional designs and interventions (e.g. instructor prompts and guiding tips, group size, grouping by gender or by viewpoints, anonymous vs non-anonymous participation, etc.); 4) evaluate and compare the effects of different communication technologies and designs (threadeded vs. linear, sinchronous vs asynchronous) on group processes and outcomes.

State Transitional Diagram: Transitional Probabilities from Two-Event Sequences*

* The circles in the diagram represent codes for specific functions of critical thinking, and the arrows represent transitional probabilities between event categories. The transitional probabilities stemming from each event do not sum to 100 because the diagram excludes six other event categories that were either occurred infrequently or were peripheral to the discussion process. Pos = Position statement, Agr = Agreement, Dis = Disagreement, Arg = Argument, Neg = Negotiation, Eval = Evaluation. This state diagran was generated by the DAT software.

Main Findings: The above transitional probability diagram illustrates some of the findings in this study:

Position statements were most often followed by arguments (33%) compared to any other types of responses.
Arguments were most likely to be followed by additional arguments or counter-arguments (49%).
Explicit disagreements to stated positions were not observed (0%). Disageements to position statements were expressed implicitly through the presentation of opposing arguments or critiques of a given position.
Arguments in support of a position were most likely to be stated in response to a disagreement (38% of responses) versus arguments posted in responses to position statements (33%). Additional results indicated that disagreements posted in response to another disagreements generate longer and more sustained discussion threads. This result supports the assumption that conflict and differences in viewpoints promote richer discussion and higher student participation.
Arguments were rarely followed by evaluation of an argment's validity, accuracy or relevancy (only 4% of the time). Instead, evaluation of arguments occurred later in the discussions as participants negotiated a group consensus. Evaluation statements occurred in 20% of the responses to negotiation statements (versus only 4% of responses to arguments).

Other Related Links:

See software tools for analyzing threaded discussions (under development)
Download this article published in The American Journal of Distance Education
More studies using event sequence analysis

Relevant Books:

Observing Interaction : An Introduction to Sequential Analysis
by Roger Bakeman & John M. Gottman (1995)
Analyzing Interaction : Sequential Analysis with SDIS and GSEQ
by Roger Bakeman & Vincent Quera

Last update: July 2003