Sequential Analysis of Group Interaction and
Critical Thinking in Threaded Discussions
Jeong, A. (2001). Sequential analysis of group interaction and critical thinking
in threaded discussions. Retrieved current date : http://bbproject.tripod.com
See research presentation in RealVideo 4/10/02
Allan C. Jeong
Department of Educational Research
Open & Distance Learning
Florida State University
This study explored and developed tools to examine student interactions and processes in threaded discussions, with particular focus on interactions that support critical thinking and rich discussion (e.g. state position, state agreement or disagreement, provide supporting arguments, evaluate arguments, define evaluation criteria, etc.). Use of the tool enabled this study to address the following questions: When a Disagreement is followed by another Disagreement or a Disagreement is followed by an Agreement, how do these different interactions affect subsequent discussions? What happens when a Position statement is followed by an supporting argument versus a Position statement followed by opposing arguments? What kinds of critical thinking can be expected to immediately follow each type of interaction (3-event sequences) and beyond? What is the average length of discussion threads that follow each interaction? When and how often do students follow-up Arguments with Evaluation and critical assessment of their arguments? Which interactions occur with probabilities statistically and significantly lower or higher than expected probabilities, or are significantly different from probabilities prescribed by existing models of critical thinking or rules of argumentation?
To examine these processes in group discussions, the Discussion Analysis Tool (DAT) was developed to perform rigorous and quantitative analysis of event sequences using the techniques of sequential analysis (Bakeman & Gottman, 1995). DAT was developed in Visual Basic using Microsoft Excel to support content analysis and measurement of event sequences and their transitional probabilities in threaded discussions - functions that are computationally intensive and infeasible without the appropriate tools. The findings of this study showed that DAT was very effective in identifying patterns in student interactions and measuring the events that follow specific interactions. For example, the probability of sustaining a discussion thread was highest when the discussion is initiated by the exchange of opposing ideas, viewpoints or disagreement. Presented arguments were rarely followed by evaluation of the arguments to test their significance, validity and accuracy. Instead, evaluations were most likely to be raised during negotiation of group concensus or drawing of conclusions. Disagreements were rarely stated explicitly in discussions. Qualitative analysis showed that disgreements were more likely to be stated implicitly through the presentation of opposing arguments and critiques.
The tools and methods developed in this study provided the means to obtain a birds-eye view of the complex processes and patterns of interaction that occur in threaded discussions. DAT also provided the means to examine group interactions, processes and associated outcomes using both quantitative measures (transitional probabilities) and qualitative descriptions (state transitional diagrams). The implications for future research are the following: 1) identify and measure unique patterns in group interactions and the outcomes that follow; 2) empirically test and identify dysfunctional as well as constructive forms of interactions; 3) empirically test the effectiveness of different instructional designs and interventions (e.g. instructor prompts and guiding tips, group size, grouping by gender or by viewpoints, anonymous vs non-anonymous participation, etc.); 4) evaluate and compare the effects of different communication technologies and designs (threadeded vs. linear, sinchronous vs asynchronous) on group processes and outcomes.
State Transitional Diagram: Transitional Probabilities from Two-Event Sequences*
* The circles in the diagram represent codes for specific functions of critical thinking, and the arrows represent transitional probabilities between event categories. The transitional probabilities stemming from each event do not sum to 100 because the diagram excludes six other event categories that were either occurred infrequently or were peripheral to the discussion process. Pos = Position statement, Agr = Agreement, Dis = Disagreement, Arg = Argument, Neg = Negotiation, Eval = Evaluation
- Position statements were most often followed by arguments (33%) compared to any other types of responses.
- Arguments were most likely to be followed by additional arguments or counter-arguments (49%).
- Explicit disagreements to stated positions were not observed (0%). Disageements to position statements were expressed implicitly through the presentation of opposing arguments or critiques of a given position.
- Arguments in support of a position were most likely to be stated in response to a disagreement (38% of responses) versus arguments posted in responses to position statements (33%). Additional results indicated that disagreements posted in response to another disagreements generate longer and more sustained discussion threads. This result supports the assumption that conflict and differences in viewpoints promote richer discussion and higher student participation.
- Arguments were rarely followed by evaluation of an argment's validity, accuracy or relevancy (only 4% of the time). Instead, evaluation of arguments occurred later in the discussions as participants negotiated a group consensus. Evaluation statements occurred in 20% of the responses to negotiation statements (versus only 4% of responses to arguments).
Complete paper to be published in March 2003 issue
in the American Journal of Distance Education