Collaboration over the Web: Strategies and Goals
|
William Brescia Heike Schaumburg Thomas M. Duffy The focus of this paper is the design of interactive learning tasks for use in distributed learning environments. We believe that the successful design of interactive learning tasks is one of the most important issues facing faculty developing instruction for distributed education. We propose that the design and delivery of distance learning environments can be placed on a continuum ranging from correspondence courses to seminars. What varies along this continuum is the interaction between learners as they struggle with and explore issues and problems in the domain. In correspondence education, there is significant up-front design effort but once designed, the course can be offered to thousands with little cost and effort. When people talk about "putting a course on the Web, " too often they are assuming the correspondence model – the Web substitutes for the postal service in delivering materials to the learners. We certainly hope the desire of faculty is to move to the other end of the continuum, toward seminar formats or at least toward formats where learners are engaged with issues in the subject matter and collaborate with classmates in working through those issues, developing and testing perspectives. We intend to illustrate effective design of learning tasks with examples from Distributed Learning Environments: Promoting Student Interaction, a graduate seminar in Instructional Systems Technology that focuses on how collaboration and meaningful learning can be fostered in a distributed learning environment. Student Collaboration for Learning: Goals and AssumptionsThe goal of this course is to engage the students in problem-solving and critical thinking about issues in the domain -- and we want to be able both to mentor and to assess their critical thinking skills. Given this goal, our first assumption is that a large class cannot work effectively as a team. Collaborative teams should have between four and ten members. With fewer than that there is not enough synergy to sustain progress - especially in a distributed learning environment. If the group is too large, it is simply too hard to be heard and too easy to hide. The second assumption is that the task should be ill-structured so that learners are working on problems that are difficult to define and for which there are many alternative solutions. It is through the ill-structuredness, through the need to constrain the problem and consider alternative solutions, that they will be engaged in the content domain. Finally, we assume that each task should have a product associated with it. That is, we never ask learners simply to discuss issues because we think that discussion without a shared purpose will not lead anywhere - at least in the typical course environment. Therefore, we always specify a product - a design, a presentation, a list of the most important issues (with their defense of that list), a report - that is to result from the collaboration. The product provides a goal orientation. With these assumptions in mind, we turn now to a description and evaluation of the tasks used in the class. Overview of the Course: Promoting Interaction among StudentsIn Distributed Learning Environments: Promoting Student Interaction we explore technology and pedagogy for promoting and supporting learner interaction. As graduate students in Instructional Systems Technology, the members of the class are all instructional designers interested in exploring pedagogical insights into distributed learning environments. Students in non-professional courses may perform differently than these highly motivated learners. The third author moderates the seminar, while the first and second authors are students in the class. Every other week the class explores a different Web site related to distributed education and Web-based collaborative learning. While the class meets face to face on campus, the work between class sessions is all completed on the Web. These Web activities are organized by teams of two class members who review the site in advance and determine what is important about the particular site what the students should learn from it. They then design a task for the class which will engage its members in that content in a way that they feel will promote meaningful learning. The task will be accomplished using a Web-based conferencing system. The facilitators also select the conferencing tool that will be used to accomplish the task. As of the writing of this paper, the class had used AltaVista Forum, Domino, ACT (developed at Indiana University-Bloomington; Duffy, Dueber & Hawley, in press), Annotate (developed at Indiana University-Bloomington by R. Eberle), and WebSmile (developed at Georgia Institute of Technology, Guzdial et al., 1997). Finally, the coordinators serve as mentors during the week, prompting students to post, giving feeback and offering suggestions. At the end of each task, each student submits a paper reflecting on the experience and analyzing its strengths and weaknesses from both a pedagogical and a technological perspective. In the next section we will briefly describe ACT, the asynchronous collaboration tool developed by the third author that was primarily used in this course, and present the rationale underlying its design. We will then describe in more detail the tasks that students engaged in and some of the findings regarding their effectiveness. In the final section we will present, as conclusions, the principles for designing collaborative tasks. Using ACT to Promote InteractionSeveral design characteristics of ACT help to support critical thinking and problem-solving. First, conferences are divided into two categories, linear and hierarchical. Each supports different kinds of discussion and, thus, calls for distinct forms of interaction. Linear conferences are better suited to promoting brainstorming, arranging for meetings and other administrative functions, and holding any discussion that resembles a short conversation. In linear discussions, the focus of the group is on the most current messages. Postings that are old and off the screen tend to be forgotten, as in a classroom discussion what someone said at the beginning of class is forgotten if not addressed by the middle of the class. In these conversations, the newest message appears as the next message in line, unconnected to any message that it is related to. With systems of this type, it is sometimes difficult to know what message this new posting is replying to. Several of the commercially available linear conferencing tools have functions that make it possible for the user to look only at new messages. In ACT unseen messages are highlighted as 'new,' allowing the user to identify unread messages at a glance. The second type of ACT conference is hierarchical. Hierarchical conferencing provides a space where the learner can actively engage in problem-solving and collaboration. The hierarchical discussion has several features that assist the learner in the process of contributing to the discussion and in developing a shared meaning with their teammates.
Figure 1: Hierarchical discussion in ACT with several different labels in use as well as identification of poster and date and time of post. In order to post a message in ACT, learners must make a "labeling" decision, thinking in advance about what kind of messages they want to post. Are they "proposing" a possible solution to the assigned task? Are they providing "evidence" that supports or contradicts a previous posting? Are they providing "reasoning" that supports or opposes a posting? The list of up to ten possible labels is under the control of the instructor. In Distributed Learning Environments: Promoting Student Interaction, the facilitating students determined the labels that they believed were most appropriate for their task. Sometimes they also involved the group in the decision on the labels. Once labels for a discussion have been chosen, they become available to users in the form of a menu. By thinking about where their posts should fit into the discussion, the learners are addressing how they are contributing to solving the problem. The goal in labeling is to promote attention to the elements of the argument as one vehicle for judging the quality of the argument, and for determining what is missing (Duffy, Dueber, & Hawley, in press). To enter a message, learners must first select a label and provide a subject to the post. Then they can enter their message in a dialog box. While posts can be of any length, certain labels carry with them the weight of longer contributions. For example, 'Questions' are usually short and ask for clarification of a particular point or direction in the conversation, while 'Rationales' by their nature are more lengthy, as learners pull in information and speculate on how it leads to a successful completion of the task. When viewing the structure of a conference, the reader is able to see several things quickly. First, the hierarchy of the postings makes clear exactly which message is in response to what posting. This also means that a user making a new posting must be careful to position it in proper relationship to the relevant posting. 'Evidence' that is posted under the wrong 'hypothesis' is useless. 'Rationales' that do not support 'evidence' will not help the group solve the task. Entering posts where they are appropriate is an important dimension of ACT. In linear conferencing systems, which put all posts at the end of the line, a learner does not have to take any responsibility for thinking about the location of his or her post in the discussion. We are interested in the learners reflecting on their participation in the discussion: we hope that, in the future, they will continue the habit of reflection on practice. We believe ACT is a powerful tool for assisting learners in constructing a learning community. Designing Collaborative Learning TasksAs mentioned earlier, one of our central assumptions is that assigned tasks should be ill-structured problems that offer a variety of possible solutions. They should require learners to analyze the subject matter and develop an approach to solve the problem in collaboration with their team members. This process is described by Fullan as "ready, fire, aim" (Fullan, 1993). In the "ready" part of the process, learners identify and research; in "fire" they grapple with alternatives and try to reach a consensus that they all can live with. In "aim," learners reflect on the process, learning experience, and final product. We will illustrate how we implemented this theoretical model with examples of tasks that learners designed in our class: Pros and Cons of Distributed EducationThe first task The first issue to be addressed was immersing the students in the issues facing distributed education to the point where they could engage in substantive discussion and think critically about them. The task was to hold an asynchronous debate. The proposition was: Resolved: Universities should embrace distributed education as a strategy for meeting the educational needs of modern society. The class was arbitrarily divided into affirmatives and negatives. Students were directed to a number of key articles addressing everything from visions of 21st century distributed education (Dolence and Norris, 1995) to using critical thinking and problem-based learning in distributed environments (Duffy, Dueber, & Hawley, in press). Students were given time limits for posts with the affirmatives going first for two days, then the negatives for two days. The affirmatives could then reply, followed by the negatives. How was technology used to support learning? The technology used for this task was ACT (see above). As explained earlier, ACT has both linear and hierarchical modes. We used the hierarchical mode for this debate. The hierarchical mode allowed the students to quickly see the progress and structure of the debate and to add their new posts where they saw a weakness for their team. Within the debate students could enter five different kinds of messages: "proposal," "evidence supporting," "evidence against," "rationale," and "questions." Successes and failures At first the negatives seemed to be somewhat reluctant to address all the issues presented by the affirmatives. At that point the exercise facilitator sent out the following message: "It [seems] that the negatives are letting several of the affirmative propositions go forward ... In the formalities of debate, the affirmatives get their points unless the negatives respond to them - and vice versa, of course." The resultant reaction can only be described as a firestorm of postings. By the end of the week there were 120 total posts. Color was used to indicate positive and negative posts: some students were strongly affected by seeing red "negative" posts below their contributions, while others were not even aware that the colors had any significance. The use of teams brought out student competitiveness. They not only wanted to win the debate, but they also worked hard to find research to support their postings, and the amount and quality of the postings was high. Several critical factors contributed to the success of this project. First, students were excited and interested by the opportunity to immerse themselves in the research in the field. Second, students were stimulated by using a new asynchronous conferencing tool. Third, the task created a feeling of competitiveness that fueled discussion. This last point was also a negative. It led some students to feel as though they had spent more time trying to win the debate than focusing on the issues. Open University (UK): A Model for Distance Education in the USA?Description of the task Open University is one of the major institutions in distance education worldwide. It delivers courses all over the world using a variety of technologies and media ranging from simple audio tapes to the Internet to joint ventures with the BBC broadcast network. The students developing a task of exploring Open University’s features paid special attention to embedding the activity in an appealing scenario that would provide complexity for the task and that would motivate the students. They set a scenario in which Ted Turner and Bill Gates as leaders of CNN and Microsoft had decided jointly to create a distance education institution in the U. S., modeled after the Open University. In the scenario, Turner and Gates invited two teams of instructional designers (our students) to develop proposals for this institution. To guide students’ exploration, the facilitators provided questions such as "What aspects of Open University are critical to its success? What are its weaknesses and how can they be avoided?" To help the students get started, the mentors also prepared a list of links to relevant Open University Web pages. After each team had submitted their initial proposals, they were asked to comment on the other team’s work. Based on these comments, both groups finally had to revise their submissions. How was technology used to support learning? AltaVista Forum (AVF) was used to collaborate on this activity. AVF is a powerful tool that supports asynchronous as well as synchronous conferencing, allows learners to share all kinds of documents and Web resources, and can be customized in many ways to the learners’ and instructors’ needs. Students in the class primarily used the asynchronous conferencing feature, although they were encouraged to try out other features if they liked. Successes and failures Overall, this task was rated positively by the students. They saw especially the second phase, when they commented on the other team’s proposal, as very beneficial to their reflection on the problem. One of the teams used the synchronous chat feature to discuss final changes to their proposal and found it to be a fast and effective medium for this purpose. Whereas after the first chat session they reported several communication problems and had difficulty coordinating their efforts (they were making changes to the document simultaneously), it is interesting to note that the second time they felt they had already started to develop strategies to make more effective use of the synchronous conferencing mode. However, several students reported technical problems with the chat. One student could not log on to the conference at all, which was frustrating for him as well as for his team members. The asynchronous conference in AVF was rated rather negatively by the students. They were confused by the complex structure of directories and entries and by the obscure navigational metaphors. Also, because in AVF only the top-level entries are shown, they felt it was difficult to get an overview of the structure of the discussion and easy to overlook responses to top-level entries. In addition, many students reported that they missed the labels they knew from ACT. They explained that these would help them structure their argument and understand the structure of the discussion when they entered the conference. Ohio University MBA Without Boundaries: Analysis of an asynchronous conferenceDescription of the task As the name implies Ohio University offers a distributed education MBA. It is a version of their residential MBA program which is based on collaborative, problem-based learning. Students work on holistic business projects in groups rather than attending discipline-based courses. The program takes two years to complete, with most of the instruction occurring in a distributed environment and face-to-face meetings held at strategic points. The task was to examine collaboration and interaction in on-line conferences that the Ohio University students were using to address the holistic projects they had been given. Students in distributed learning environments were divided into two teams. Each was charged with producing a comparative analysis report addressing questions like "What evidence is there of problem solving activity? What evidence is there of collaboration? What is the role of the faculty? Are the faculty displaying characteristics of successful mentors? Are the team members listening to each other? and Is there evidence of improvement over time in each of these categories?" Three research papers, by the faculty members at Ohio University who developed the problem-based program, were made available (Milter & Stinson, 1995), and a more general Problem Based Learning paper by Savery and Duffy (1996) was provided. How was technology used to support learning? The OU/MBA students used Lotus Domino, a Web-based conferencing tool. Students in Distributed Learning Environments were able to observe archived tasks that had been completed by OU/MBA students. In most cases, this was the students’ first exposure to Lotus Domino. Discussion was again carried out in ACT. Labels that were available to students were "alternative," "concern," "example," "larger context," "more detail," "plan," "proposition or hypothesis," "question," and "rationale." The discussion was begun by the student leaders posting the questions listed above. Successes and failures Learning Lotus Domino was difficult for some of the students, who needed help from the student leaders to be able to access the necessary information. The major difficulty that students soon discovered was that the on-line conferences did not contain all the discussion. There were references to face-to-face discussions which were not explained on-line. While everyone at OU clearly knew what was going on, it was impossible to divine as an outsider. This and the sheer amount of data made it problematic for the students to discover examples to answer their task questions. Once students developed a search schema, finding characteristic posts was easier. Students in our class felt that having a theoretical basis made it possible to observe the OU students attacking a problem and seeing the distinct parts such as collaboration, problem solving, and interactivity. After the student leaders sent individual messages to the team members, postings increased. While totals of posts should not be used as the only indicator of success of a task, the fact that the two teams made 177 posts - more than 50 posts above the total for any other task - does show engagement in the activity. Analysis of the content of the posts showed that the students were using the labels properly and were submitting messages that contained information drawn from the data set that clearly addressed the questions in the task. The postings relating to rationale were thoughtful and based upon research. New Jersey Institute of Technology: Improving a group decision processDescription of the task The New Jersey Institute of Technology (NJIT) Web site offers a rich database of documentation about the development, utilization, and evaluation of the Virtual Classroom, an elaborated distributed learning environment designed at NJIT. The goal of the mentors designing a task related to this Web site was to engage the students in exploring and critically evaluating the features of the Virtual Classroom by analyzing the articles provided on the NJIT Web site. The students were asked to evaluate the materials from the perspective of instructional designers making suggestions for how the effectiveness of collaborative decision-making could be improved. As a final product, they were required to develop a prioritized list of recommendations including a rationale. For this task, the entire class (eight students) worked together as one team. How was technology used to support learning? Two tools were used to perform this task: Students used the hierarchical conference mode of ACT for the initial discussion. Labels available for the students were: "alternative," "evidence supporting," "evidence against," "proposal," "question," "rationale," "recommendation." For composing the list of recommendations, they used Annotate, a collaborative writing tool. Annotate is a Web-based tool that has been developed recently at Indiana University by Ruth Eberle. It allows users easily to annotate and comment on an electronic copy of each other’s documents. Successes and failures During their time as class leaders, the students made a serious effort to focus on mentoring using e-mail messages, both to the group as a whole and directly to individuals. These messages were, on the whole, of a positive nature and concentrated on specific activities that needed to be started to move toward completion of the task. They took special care to try to send messages that were designed to create a psychological climate of trust which would encourage the students to share and reflect upon their work on the task. Some were also more specific in asking students to provide detailed information from particular articles to ensure that the student’s rationales were based upon accurate and sufficient knowledge. They also mentored students to tighten their focus on particular themes and to share their thinking with regard to how the task might be completed. These messages were received positively by the class and seemed to have an effect on the use of ACT. However, even though students appreciated the fact that someone was paying attention to what they were doing, they did not always agree with the advice they received and therefore did not always follow the mentor’s suggestions. Nevertheless, we think the important point is that in a distributed environment it is necessary for the instructor to be present in as many ways as possible, and mentoring is one key way to make your presence effective. It appears that students like to know their work is being monitored and to receive feedback on their progress. They just do not always pay attention to that feedback. As it turned out, a problem of this task was that, from the materials given, the students had difficulty understanding the actual collaboration in the Virtual Classroom. Although the environment was described in several articles, the students felt that they needed more detailed descriptions (if not direct experience) of the learning environment to be able to evaluate the Virtual Classroom and make suggestions for improvement. Therefore, the discussion started very slowly, frustrating students who had posted early and did not get a response from their classmates. However, after the students had overcome the first puzzlement of not finding the information they had expected in the literature available to them, they used the material that was there, and a vivid and productive discussion about the Virtual Classroom developed on ACT. Because the mentors wanted the students to condense their results in the form of a collaborative list of recommendations, they interrupted this discussion after a couple of days and prompted the students to move over to the collaborative writing tool. The students followed this recommendation, but, instead of working on a collaborative paper, they used the annotation structure to continue their discussion. This unexpected use of Annotate might have been due to the fact that the mentors had interrupted the discussion at a point where the students were not ready for a conclusion. Also, it was the first time that the class had used Annotate, and the students were still experimenting with how to use this tool. Conclusion- There is a wide range of acceptance of asynchronous conferencing. Some students adopt it freely, some long for face-to-face contact, and still others will be happy with synchronous chat sessions. Some mix of the three seems appropriate. - Developing ill-structured problems can be a tricky business. It is sometime difficult to determine an appropriate time to allot to a particular task before beginning it. - Synchronous and asynchronous conferences have different strengths. Synchronous conferences are better suited to promoting brainstorming, arranging for meetings, and carrying out other administrative functions. They are also useful in decision-making. Asynchronous conferencing provides a space where the learner can actively engage in problem-solving and collaboration. Asynchronous conferences give the learners an opportunity to find relevant research and reflect on their interchanges of information.. - You will have to find your own on-line mentoring style. For a long time we have known that e-mail messages are sometime misunderstood because writers are too casual about what they say, and readers misconstrue the intent. Providing support, asking for classifications, correcting errors, and modeling take on different aspects on-line. Take some time to consider what your role as mentor should be. Take some time for students to talk about their roles as mentors and protégés. References
|
