Paper to be presented at AECT October, 2003
Dr. Allan Jeong
Florida State University
This study examines techniques for structuring online discussions and their effects on group interaction patterns that support critical thinking. With Message-Reply Structures, students can only post specific types of replies depending on type of message (e.g. argument, evidence, critique, explanation). With Time-based Structures, types of messages can only be posted during certain periods or times of the week. Their effects are compared with Unstructured Discussions with no constraints on where or when messages/replies are posted.
Computer conferencing systems have been developed to scaffold critical thinking and argumentation (Sloffer, 1999; Turoff, 1999; Jonassen & Cho, 2002) - building and evaluating arguments to support a position by weighing evidence and counter-evidence. These systems impose constraints and structures that determine where, when and what messages can be posted. A common technique for imposing constraints on what messages can be posted to a discussion is Message Labeling, when students select and compose messages/replies to address specific tasks or functions. The systems display the posted messages with labels to enable participants to easily identify the functions of all posted messages. For example, Duffy, Dueber & Hawley (1998) present a constraint-based argumentation scaffold that restricts students to posting four types of messages (argument, evidence, critique, explanation). This constraint-based technique can improve the visualization and understanding of task-related mental structures (Derry, 2000) and complex discourse arguments to decrease task difficulty (Cho & Jonassen, 2002), support better problem-solving and learning (Turoff & Hiltz, 1998), and more specifically, enable participants to locate disagreements and conflicts to engage in more sustained and deeper discussions to resolve conflicts (Jeong - work in progress).
To provide guidance on when and where to post messages, Remidez & Jonassen (2002) developed a system that imposes Message-Reply structures in threaded discussion boards. Their system not only imposes message-labeling constraints with message labels pre-determined by the instructor, but also imposes rules on the types of responses that can be posted to each type of message. For example, evidence and explanations can only be posted in reply to arguments, and critiques can only be posted in reply to evidence. Suthers et al (2002) developed a system in which replies must be labeled with three possible links (for, against, and). Even with these built-in structures, many distance educators may already be applying similar structures (Time-Based Structures) by breaking discussions into specific phases or periods for performing specific functions and tasks. The purpose for imposing types of structures is to prescribe particular processes or models of argumentation, problem-solving, or even instructional design to improve the quality and outcomes of group discussions and computer-supported collaborative work.
At this time, no studies provide empirical evidence to determine the true advantages and disadvantages of these structures. The purpose of this study is to examine how these structures affect the quality and quantity of interactions between discussion participants, particularly interactions that encourage critical thinking. Using a controlled experimental design, this study will test the effects of the two structures against a control condition in which no structures are imposed on group discussions while implementing message labeling in all three conditions. Some of the measures on which the conditions will be compared are the following: 1) the percentage of arguments that receive replies with supporting evidence, explanations and information; 2) the percentage of arguments and evidence that are challenged and questioned versus the those that are left unchallenged; 3) the percentage of arguments and evidence that are evaluated on their accuracy, validity and significance; 4) the percentage of messages that receive responses versus no responses; and 5) the average length, depth and breadth of observed discussion threads.
The participants are graduate students enrolled in an online course in a Masterís program in Distance Education, with ages ranging from 20 to 40 years. In all three experimental conditions, students engage in a series of five or more online debates in a Blackboard threaded discussion forum on issues related to weekly topics of study. For each weekly debate, participants are randomly assigned to support or oppose a given position on an issue. The number of participants in each condition will be ten or more students in order to maintain a critical mass for generating sufficiently different viewpoints to drive the discussions (Winograd, 2002). All postings to the debates are constrained by message labels, so student must post messages to address a specific function (e.g. argument, evidence, explanation, critique, etc.). Students are also required to manually label their messages within the message titles with labels corresponding to each function (e.g. ARG, EVID, CRIT). Students must post a minimum of four messages per week with participation counting for 20% of the course grade.
Previous studies that examine interaction patterns and critical thinking in online discussions relied on content analysis (Gunawardena, 1996; Newman, 1995; Levin, Kim & Riel, 1990). This method simply counts the frequency of observed critical thinking events to compare treatment groups and describe group interactions. This study is unique because interaction patterns in online discussions will be examined using event sequence analysis (Bakeman & Gottman, 1997; Jeong 2003) - a method that measures how likely a message will receive a response and the probability (transitional probability) of observing a specific type of response for a specific type of message. The theoretical justification for focusing on the transitional probabilities between messages and threaded responses are based on assumptions in dialogic theory (Bakhtin, 1981) which posits that all possible meanings of a word interact, possibly conflict, and affect future meanings. Meaning is produced by the relationship between one utterance and another, and not by the utterance alone. Therefore, meaning is affected, re-negotiated and reconstructed as a result of conflict arising from social dialogic reasoning. To perform the tedious event sequence analysis, this study replies on a computer program (Jeong, 2002) to download, format and process Blackboard forum messages and compute transitional probabilities between messages and responses.
This will be the first study to empirically test the effects of Message-Reply
and Time-based structures on group interactions and critical thinking in
online discussions. Using the method of event sequence analysis - a method
that no other researcher has yet applied to this field of research - might
explain why this study is the first of its kind to conduct such an empirical
test. Focusing on the relationships between messages and responses will
provide the much needed framework and approach for generating quantitative
and empirical evidence and future studies to determine the best structures
and instructional strategies for enhancing group interaction and outcomes
in online discussions.