Document View

Lou et al quantitatively synthesized the empirical research on the effects of social context when students learn using computer technology. The results indicate that, on average, small group learning had significantly more positive effects than individual learning on student individual achievement, group task performance, and several process and affective outcomes.

Computer technology (CT) and the tremendous growth of information technologies are transforming the world and the way education is conducted. Electronic data processing, information systems, graphic designs, and computer-mediated communication are making the computer an increasingly indispensable tool in nearly every aspect of work and life. In schools, students are using CT to facilitate their learning in various subjects as well as to acquire CT knowledge and skills to meet the challenges in this rapidly changing technological and information age. For example, in mathematics and science, educators and scientists are beginning to worry that school learning cannot keep pace with the developments in science, and they suggest using CT to help fill the gap (Molnar, 1997). More efforts than ever before are being made by governments and institutions to introduce and integrate computers in schools. It is estimated that over 4.4 million computers are currently installed in America's classrooms and the ratio of students to computers has dropped from 125 students per computer in 1984 to the current ratio of 10 students per computer (Coley, Cradler, & Engel, 2000).

Although CT has the potential to be a powerful and flexible tool for learning (Scardamalia & Bereiter, 1996), past experiences with the integration of older technologies into schools (e.g., radio, television, early computer-assisted instruction) emphasize that merely installing the hardware does not produce the desired outcomes (Clark, 1983). Successful and effective learning with CT must rely on sound instructional strategies (Albright & Graf, 1991; Coley, Cradler, & Engel, 2000). One of the instructional strategies concerns social context; specifically, whether students learn with CT individually (i.e., with one computer per student, each working on his or her own task) or in a group (i.e., with two or more students per computer on the same task in a face-to-face setting, or two or more students collaborating on the same task synchronously or asynchronously over a distance).

Historically, the most common instructional strategy was to have students work individually at a computer. When Skinner (1961) invented his first teaching machine, it was designed to individualize instruction using principles of operant conditioning through careful sequencing of the instruction and appropriate reinforcement. During the 1960s, popular programs such as Individually Prescribed Instruction and Keller's Personalized Systems of Instruction influenced the trend toward individualized use of computers. The rationale was that learning would be facilitated when instruction could be adapted to the students' individual differences (e.g., prior knowledge, interests, and learning styles). CT with its flexible sequence, interactivity, and feedback made individualized instruction possible. Therefore, during the 1960s and 1970s, when the computer was first introduced to schools, computer-assisted instruction (CAI) was usually designed in the form of drill-andpractice activities and was used to individualize student learning. It was hoped that CT would enable each learner to work at his or her own pace, on materials at his or her own difficulty level, and would provide immediate feedback for what he or she had done.

The initial expectation that CT would revolutionize education, however, was not realized for several reasons (Means, 1994). First, CT was not adequately advanced and flexible at the time; the machines in use were slow, not very powerful, and not easy to use. Second, in terms of the instructional design, the computer programs were mostly text-based drill-and-practice (Kulik & Kulik, 1986) and, therefore, limited in terms of meeting a broad range of pedagogical activities and learning goals. Third, many teachers feared that they would be replaced by machines, and especially those without adequate training often avoided using them outside of special computer lab activities with the computer teacher. Finally, many teachers and parents feared that individual learning with computers might produce "social misfits" (Crook, 1994) who by working alone would be devoid of the social skills normally part of the regular classroom routine.

Since the 1980s, with the widespread appearance of the microcomputer and its ever-increasing power, capabilities, and lower prices, there has been a renewed enthusiasm for integrating CT in education. Various types of computer programs have been designed and used in schools. The earlier single type drill-and-practice program has now been expanded into a greater variety: microworlds, intelligent tutorials, simulations and games, interactive hypermedia and multimedia environments, computer-mediated communication, and Web-based courses.

Another difference from the earlier use of computers in schools is that students are often assigned to work in small groups for several reasons (Jackson, Fletcher, & Messer, 1986, 1988). First, few classrooms have sufficient technological resources to afford all students individual access to computers at will. Thus, there are practical and economic constraints which affect student access and encourage teachers to find ways for students to use technology together as a tool for learning. Second, several theories (e.g., constructivism, socially shared cognition, distributed learning, and so forth) and empirical investigations support the concept that students learn well together. For example, peer collaboration, exposure to multiple perspectives, and so on can be important processes in the learner's construction of knowledge. In other words, regardless of practical constraints, it may be advisable for students to collaborate when using CT for learning.

Considerable research has been conducted since the mid-1980s investigating the effects of social context when learning with CT. The results, however, are not consistent. Some researchers found support for group learning. For example, Johnson, Johnson, and Stanne (1985, 1986) found that cooperative group learning could overcome the social isolation commonly associated with individual learning with CT and that students learning in small cooperative groups achieved more than students in the individual condition. However, these findings on the effects of group learning were not consistently supported by other research results. In a narrative review of 20 studies comparing small group learning with CT and individual learning with CT, Shlechter (1991) found that the collective evidence was not clear. The research reviewed indicated no consistent effects for either small group or individual learning on students' academic achievement or retention scores.

The unclear nature of the effects of social context when students learn with CT and the fact that considerably more research has been conducted on the topic since Shlechter's (1991) review calls for a more systematic and up-to-date integration of the literature both for theory development and for pedagogical guidance. We believe that learning with CT may represent different circumstances and contexts in which learning occurs than learning without CT presents. For example, mouse and keyboard control may affect the nature of learning dynamics; it cannot be assumed that the quality and quantity of collaborative learning experiences with CT are necessarily the same as when CT is absent. Furthermore, the apparent inconsistency of the study results on the effects of social context when learning with CT suggests that the context for effective learning with CT may not simply be a question of small group versus individual learning. Some characteristics inherent in the studies, such as technology and task design or group learning strategies, may mediate the effects of social context.

The purpose of this review is, therefore, to conduct an extensive meta-analysis of the empirical literature on small group versus individual learning with CT. Specifically, this meta-analytic review seeks answers to the following questions:

1. Does small group learning with CT enhance student achievement and other outcomes, compared to individual learning with CT? If so, to what extent?

2. What study features moderate the effects of social context in learning with CT? Is the moderating influence of study features similar across different outcomes?

3. What are the optimal conditions for effective small group learning with CT? For example, when should it occur and what type of small group learning facilitates better learning with CT?

4. Are there any conditions in which individual learning with CT may be more effective? For example, what design characteristics of computer programs facilitate better individual learning?

In the following sections, we review findings in two areas of research, that is, the research on learning with CT and the research on small group learning, to help identify features to consider in our own quantitative integration of the effects of social context on learning with CT.

Types of Programs

Over the last five decades, a variety of computer programs have been developed and used to support student learning: from early mainframe-based or microcomputer-- assisted instruction (CAI) or computer-based instruction (CBI), to Logo, simulations, hypertext, computer-mediated communication (CMC), and the Internet. Guided by different learning theories, philosophies, or developments in technology, each type of program appears to have distinct characteristics, purposes, and different ways to facilitate student learning. Means (1994) classified various types of learning with CT into four main categories: tutor, exploratory environment, tool, and communication media. Tutoring programs are used to directly teach students by providing information, demonstration, and practice opportunities. Examples of tutor programs are tutorials or practice CAI. Exploratory programs are used to encourage active student exploration and discovery learning. Examples of exploratory programs include microworlds (e.g., Logo), simulations, and hypertext-based or hypermedia-based learning environments. Tool programs refer to the general-purpose technological tools such as word processing, spreadsheet, and data-analysis software, which are used to accomplish tasks such as writing, data storage, and data analysis. Computermediated communication media include e-mail, computer-conferences, computersupported-collaborative learning (CSCL) systems, and the Internet, which allow groups of teachers and students to communicate and share information electronically, to learn and to collaborate across distance.

Extensive research has been conducted on the effects of learning with CT. The results of several meta-analyses (Kulik, Kulik, & Cohen, 1980; Kulik, Bangert-- Drowns, & Williams, 1983; Bangert-Drowns, Kulik, & Kulik, 1985; Samson, Niemiec, Weinstein, & Walberg, 1985; Kulik & Kulik, 1986, 1991; Niemiec, Samson, Weinstein, & Walberg, 1987; Ryan, 1991; Fletcher-Flinn & Gravatt, 1995; Fazal, 1996) have generally indicated overall positive effects of learning with CT on student achievement, attitudes toward learning, and self-concept as compared to traditional instruction. However, other quantitative and narrative reviews indicate that the effects of learning with CT appear to differ for different types of programs.

Niemiec, Samson, Weinstein, and Walberg's (1987) meta-analysis on the effects of learning with CT at the elementary school level found that the effects appeared greater for drill-and-practice programs (mean ES = +0.47) and tutorials (mean ES = +0.34) than for problem solving (mean ES t= +0.12) programs. Similar results were found by Kulik and Kulik (1991) at pre-college levels but not at post-secondary levels. They found that at pre-college levels, CAI (mean ES = +0.36) appeared more effective than CEI (i.e., computer-enriched instruction, which is similar to exploratory and tool programs defined by Means, 1994). The mean effect size for the latter was not significantly different from zero. But at the post-secondary levels, the mean effect sizes for CAI and CEI were both significantly positive (mean ES = +0.27, and +0.34, respectively).

Sivin-Kachala and Bialo (1994) reviewed research on the effectiveness of learning with technology in schools during 1990-1994. They found that tutorial and tool programs produced differential achievement gains in mathematics for high school students. While those using the tutorial program demonstrated higher achievement in computational skills, those using the tool program achieved higher scores in conceptual understanding.

Reeves (1998) summarized and organized the evidence on the effects of using technology for learning in two categories: learning "from" technology (i.e., technology as a tutor) versus learning "with" technology (i.e., technology as a cognitive tool or exploratory environment). The review suggests that the greater value of technology-based tutors was in its ability to motivate the students, decrease instruction time, and increase equity of access to quality instruction. In contrast, the greater value in using technology-based cognitive tools such as databases, spreadsheets, expert systems, and communication software was the learners' engagement in real world tasks such as exploring, analyzing, and interpreting information, solving complex problems, and communicating effectively what they knew to others. These tools enabled the learners to take active control of their learning, and to construct knowledge rather than to reproduce it.

Similar conclusions were reached by Coley, Cradler, and Engel (2000), who surveyed the status of CT use in schools. Based on their review, the authors concluded that drill-and-practice forms of CAI are effective in producing achievement gains in students and that although more pedagogically complex uses of technology generally show more inconclusive results, many offer promising and inviting educational vignettes.

Other Technology and Task Design Characteristics

Computer programs also differ in a number of other technological design features. Sivin-Kachala and Bialo's (1994) review described four major instructional software design characteristics that significantly affected student learning. These four characteristics were instructional control, type of feedback, embedding of cognitive strategies, and inclusion of animated graphics. Studies on instructional control showed that students learning under mainly learner-control conditions outperformed those learning under mainly system-control conditions. Studies on feedback showed that students working with programs that provided feedback performed better than those working with programs that provided no feedback and that those receiving adaptive feedback performed better than those receiving static feedback. Other studies on cognitive strategies found that embedding cognitive strategies such as repetition, rehearsal, paraphrasing, outlining, cognitive mapping, and drawing analogies and inferences in computer programs facilitated student learning. Studies with animated graphics in reading and physics found that the use of animated graphics significantly increased achievement or reduced the necessary time on task.

Azevedo and Bernard (1995) conducted a meta-analysis of 22 studies on the effects of different types of feedback. They found large positive effects of feedback on student learning when measured by immediate achievement tests (mean ES = +0.80) and moderate positive effects when measured by delayed posttests (mean ES = +0.35). They also found that students receiving feedback that verified not only the correctness of the learner's answer but also the underlying causes of error achieved significantly higher than students receiving evaluative feedback only.

Davie and Inskip (1992) studied the effects of designing fantasy role-plays, providing pre-structured databases, and involving guest visits in a computer-mediated distance learning course in literature. Their qualitative research results suggest that these instructional design strategies promoted the success of their CMC course. The authors, therefore, argue that the success of CMC courses depends on creative instructional design to support active learning and participation.

Lundgren-Cayrol (1996) studied the effects of different levels of facilitator intervention in computer conferences that supported an undergraduate distance learning course in educational technology. She found that different levels of facilitator intervention had differential effects on student learning. Those who learned under the higher level of intervention achieved significantly higher than those who learned under the lower level of intervention.

Small Group Learning Strategies and Task Structure

A variety of group learning strategies are employed when students learn in small groups. In some studies, specific cooperative learning strategies were used to ensure positive interdependence and individual accountability; in other studies, students were generally encouraged to work together; and in still other studies, there were no specific strategies employed at all, beyond the physical placement of learners together and the lack of prohibitions on collaboration.

Johnson and Johnson (1989) conducted a meta-analysis of studies comparing classrooms using cooperative learning approaches versus those using competitive or individualistic approaches. Their results indicate that students in the cooperative condition learned significantly more than those in either the competitive condition (mean ES = +0.67) or the individualistic condition (mean ES = +0.64). Cooperative learning strategies also produced medium to large positive effects on student attitudes toward the subject matter and learning, liking of other students, feelings of social support, and self-concept.

Slavin (1989) conducted a meta-analysis of cooperative learning studies using his "best evidence" approach. His review showed a small positive effect of cooperative learning on student achievement (median ES = +0.21). He also found that students learned significantly more in groups where both positive interdependence and individual accountability strategies were used than when either one was used alone.

When working in groups, students may work on a variety of tasks. Some tasks may be ill-structured and open; others may be highly structured and closed. Cohen's (1994) review of small group learning found that groups were not productive when tasks were closed with only one fixed answer to the question; groups were more productive when tasks were open to multiple perspectives and solutions. Cohen argued that in the former case, extended group discussions may not be necessary; whereas in the latter case, open exchange and elaborated discussion are necessary to facilitate conceptual learning through cognitive dissonance and elaboration.

More recently, Lou et al. (1996; Abrami et al., 2000; Lou, Abrami, & Spence, 2000) conducted a meta-analysis on the effects of within-class grouping (including both cooperatively structured groups and non-structured groups) versus whole class instruction. Their results showed that, on average, there is a small positive effect of within-class grouping over whole class instruction on student achievement (mean ES = +0.17). However, the results also showed that there was significant heterogeneity in the effect sizes analyzed. Through study features analyses, they identified a few study features that accounted for the significant variability across the findings. The substantive moderators include: group learning strategy, group size, grouping basis, amount of teacher training in the cooperative learning methods, and adaptation of instructional material and methods to small group learning. They found that students learned more under cooperative outcome interdependence than when no such structure was in place; small groups of three to four members were more effective than larger groups; group learning was most effective when grouping was based on mixed criteria rather than on ability alone; and teacher training in and experience with small group instructional strategies and adaptation of instruction methods and materials helped maximize student learning in small groups.

Learner Characteristics

The literatures on both technology-supported learning and small group learning suggest that the effects of learning with technology or in small groups may depend on characteristics of the learners such as computer experience, gender, grade level, and ability levels. Jackson, Fletcher, and Messer (1988) studied the effects of experience on microcomputer use in primary schools. The results of their study showed that learners' experience with CT was an important factor. They found that inexperience with computers often caused computer anxiety or computer phobia, which tended to exaggerate the difficulty level of a computer task.

Similar findings were observed by other researchers. When studying the effects of networked computers on class discussion, Bump (1990) reported that the initial lack of knowledge about the computer system stressed the students. The author reported that students felt frustrated and that they required time to gain ease in the use of the system. Bridwell, Sirc, and Brooke (1985) also found that experience with computer programs influenced the effects of using word processors for writing.

Niemiec, Samson, Weinstein, and Walberg's (1987) meta-analysis of the studies conducted in elementary schools indicates that CAI (particularly drill-andpractice programs) was most effective for lower ability students and for students at lower primary grades, especially when tasks were simple, involving paired association such as vocabulary acquisition and mathematical computation. On the other hand, Roblyer, Castine, and King's (1988) meta-analysis found that the mean effect size for low-achieving students (mean ES= +0.45), although somewhat higher, was not significantly different from that for regular students (mean ES = +0.32).

Some researchers have studied gender differences among students learning with computers. While the common belief is that male students learn more from computers, Roblyer, Castine, and King's (1988) review of 10 studies that provided separate results for males and females indicated no significant differences between males and females in student achievement. Results for student attitudes toward computers revealed a nonsignificantly higher mean effect size for male students (mean ES = +0.29) than for female students (mean ES = +0.05).

Fletcher-Flinn and Gravatt (1995) conducted a meta-analysis of 120 CAI studies published between 1987 and 1992. They found that the effects of learning with CAI appeared highest for kindergarten and preschool (mean ES = +0.55), followed by elementary school (mean ES = +0.46), then high school (mean ES = +0.32), then college/university (mean ES = +0.26) and finally, adults in training situations (mean ES = +0.22).

Lou et al.'s (1996) meta-analysis of within-class grouping found that small group learning had differential effects for students at different relative ability levels. Although the mean effect sizes were positive for all ability levels, group learning was more effective for lower ability learners than for medium ability learners. In addition, they found that different group ability composition had differential effects for students at different ability levels. Lower ability students learned more in heterogeneous groups, whereas medium ability students learned more in homogeneous ability groups. For high ability students, there was no significant difference whether they learned in heterogeneous or homogeneous groups. Lou et al. suggested that low ability students may gain most when they have more able peers to provide them with timely and elaborated assistance and guidance; high ability students may benefit from providing those elaborated explanations. Medium ability students, however, may not benefit from heterogeneous groups when they neither give nor receive explanations. Homogeneous ability grouping may be better for medium ability students because they may share in giving and receiving explanations among themselves. In addition, Lou et al. suggested that homogeneous grouping may benefit from group cohesiveness since students may share similar expectations about group goals. Medium and high ability students may especially benefit from homogeneous grouping without compromising their aspirations or pace of learning to accommodate the lower ability students.

In summary, the research reviewed on learning with CT indicates that although it has generally positive effects, the effectiveness of learning with CT is significantly related to several characteristics such as type of programs, feedback, learner control, computer experience, and ability levels. Similarly, the research on small group learning indicates that although it in general has positive effects on learning outcomes, the effectiveness of small group learning is significantly related to several characteristics such as cooperative learning strategies, task structure, teacher training, group size, group composition, and ability levels. These findings have important implications for the initial design of the present meta-analysis on the effects of small group learning with CT. It is possible that both sets of factors may influence whether small group or individual learning may be more effective when learning with CT. We therefore included them in our attempts at identifying the moderator study features used in this meta-analysis.

Method

This meta-analysis quantitatively integrates the findings from primary research on the effects of social context when students learn with CT. The procedures employed to conduct the quantitative integrations are outlined below under the following headings: identification of studies, outcomes and study features coding, effect size calculations, number of findings extracted, and data analyses.

Identification of Studies

Studies included in this meta-analytic review were first located through a comprehensive search of the literature. Electronic searches were performed on the ERIC (1966-1999), PsycLit (1974-1999), and Dissertation Abstracts (1965-999)1 databases. Although the search strategy varied depending on the database, search terms included: computer* and any terms related to small group learning such as cooperative or collaborative learn*, or small group*, or team*. Through branching from primary studies and review articles, other citations were identified.

To be included in this meta-analysis, each study had to meet all the following inclusion/exclusion criteria:

1. The study had to involve situations where students learned using computers (i.e., students were directly involved in using computers for learning, whether learning CT skills or using CT to learn other subjects).

2. The study had to have employed an experimental design which allowed for the comparison of small group learning with CT versus individual learning with CT. More specifically, the investigation of social context meant comparing learning with computers in small groups (i.e., with two or more students per computer on the same task in a face-to-face setting, or two or more students collaborating either synchronously or asynchronously on the same task electronically) versus learning with computers individually (i.e., with one computer per student, each working on his or her own task).

3. The minimum group size was 2 and the maximum group size was 10. (Ten was used as an inclusion criteria when coding the studies. However, the largest group size found in any of the studies was 5).

4. The study had to report cognitive outcomes, process measures, or affective outcomes for both experimental and control groups. Different types of outcomes were coded and analyzed separately (see the section "Outcomes and Study Features Coding" for the types of outcomes coded and analyzed; some outcomes were dropped due to small sample sizes). Studies with insufficient data for effect size calculations (e.g., with means but no standard deviations or no inferential statistics) were excluded.

Using the above inclusion and exclusion criteria, abstracts from electronic searches, references from primary studies and review articles were examined to identify potential studies for inclusion. If there was doubt, the study was collected. Next the collected studies were read independently by two researchers for possible inclusion. Any study that was considered for exclusion by one researcher was checked by the other. One hundred and twenty-two studies met all the inclusion criteria.

Outcomes and Study Features Coding

The purpose of coding outcomes and study features was to identify those methodological and substantive characteristics that may be responsible for significant variations in the findings. Three steps were followed in coding the studies. First, based on the review of the related literature, a broad coding scheme was developed outlining four categories of substantive study features that might interact with the effects of social context in learning with CT. These four categories were technology, task, grouping, and learner characteristics. In addition, outcome and methodological features were also included in the coding scheme.

Next, using the broad scheme as a framework, a random sample of 25% of the primary studies was nomologically coded to identify salient study features in the literature as well as salient categories within each study feature so as to avoid researcher bias (Abrami, Cohen, & d'Apollonia, 1988; Abrami, d'Apollonia, & Cohen, 1990). As a result of the nomological coding, the original coding scheme was revised and developed into a codebook. Outcomes and features with more than three occurrences in the sample were included in the codebook.

Table 1 describes individual achievement, group task performance, and several learning process and affective outcomes extracted and analyzed in this review. Individual achievement and group task performance were coded and analyzed separately in this meta-analysis as a result of our preliminary analysis which showed that the two outcomes were significantly different not only in their mean effect sizes but also in the factors moderating the relationship with social context. The analysis that used individual achievement as the outcome compared the achievement scores of those who learned in small groups versus those who learned individually on individually administered immediate or delayed posttests. The analysis that used group task performance as the outcome compared group performance versus individual performance during task realization. Thus, the analysis that included group task performance explored the relationship between social context and performance where students learning in groups completed a group task and where students working individually completed an individual task.

Process measures included frequencies of positive peer interaction, interactivity with computers, request help from teachers, task completion time, task attempted, use of strategies, perseverance, and success rate. Affective outcomes included student attitudes toward computers, subject or instruction, group work and classmates, and academic self-concept.

Table 2 describes the 30 methodological, outcome, and substantive study features coded for each study. Methodological features included student equivalence, publication status, and publication year. Outcome features included type of outcome, outcome measure source, outcome measure time, and whose outcome. Substantive features were coded in four categories: technology, task, grouping, and learner characteristics. Technology characteristics included type of programs, design orientation, feedback, instructional control, teacher support, and setting of collaboration. Task characteristics included subject, type of tasks, task structure, task familiarity, and task difficulty level. Grouping characteristics included group composition, presence of others, group learning strategy, group work experience or instruction, group size, amount of peer interaction, number of sessions, and session duration. Learner characteristics included grade level, relative ability level, gender, and computer experience.

Finally, the coding was performed by two coders independently. Their initial coding agreement was 80.55%. Disagreements between the two coders were resolved through discussion and further review of the disputed studies.

Effect Size Calculations

The basic index for the effect size calculation is the mean of the experimental group minus the mean of the control group divided by the pooled standard deviation (PSD). That is, the effect size is a measure of the superiority of learning with computers when working in a group versus working alone. The main reason for using the PSD is that the assumption of homogeneity of variance in the population is often reasonable, in which case the PSD is more stable and provides a better estimate of the population variance than the control group SD alone (Hedges & Olkin, 1985; Hunter & Schmidt, 1990; Rosenthal, 1991). Another reason for the choice of the PSD is that estimated effect sizes based on incomplete results (e.g., t values, F values, ANOVA tables, or p levels) are more readily comparable to effect sizes calculated in this manner.

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

In studies that report post-test data only, we used the post-test mean difference in the numerator and the post-test PSD in the denominator. In studies that provided gain scores or both pre-test and post-test data, we used the gain score difference in the numerator to control for pre-test differences, but the post-test PSD was used in the denominator rather than the gain score PSD. The gain score PSD is usually smaller than the post-test PSD (Glass, McGaw, & Smith, 1981), and therefore estimates of effect size tend to be larger when gain score PSDs are used. When the post-test SDs were not provided in the study, we tried to estimate the post-test PSD. Such estimation requires knowing test reliability, which is, unfortunately, not usually reported in studies. In several cases, we had to estimate a "typical" reliability for that class of measures based upon our knowledge of the literature. Specifically, we estimated r = .85 for standardized tests and r = .75 for unstandardized tests.

Effect sizes from data in the form of t value, F value, p level, frequencies, r value, etc. were computed via formulas provided by Glass, McGaw, & Smith (1981) and Hedges, Shymansky, & Woodworth (1989). For studies that reported only a significance level, effect sizes were estimated. When the direction of the effect was not available, we used an estimated effect size of zero. When the direction was reported, a "midpoint" approach was taken to estimate a representative t value (i.e., midpoint between 0 and the critical t value for the sample size to be significant) (Sedlmeier & Gigerenzer, 1989).

Formulas for calculating effect sizes were entered into an EXCEL (Microsoft, 1997) spreadsheet. Raw data for each finding were extracted by two researchers separately and then checked for reliability. The initial agreement between the two researchers was 93%. Disagreements were subsequently resolved through discussion and further review of the disputed study findings.

Number of Findings Extracted

There are generally two major approaches regarding the number of findings to be extracted from each study: a single finding per study or multiple findings per study. The advantage of extracting only one finding per study is that the assumptions of independence are met. However, a major problem with this approach is that the differences within a study between different categories of subjects (e.g., males and females), or between different treatments under investigation (e.g., groups using specific cooperative learning strategies versus groups that were only generally encouraged to work together), or between different outcome measures (e.g., achievement and task performance) are lost.

Multiple effect sizes extracted from a single study, on the other hand, can be problematic because methods of research integration normally assume that effect sizes are independent. Non-independence can increase Type I or Type II error rates (Glass, McGaw, & Smith, 1981). The problem of dependence was resolved in the following three ways in the present meta-analysis. First, findings for each outcome were analyzed separately. Only one finding per outcome was extracted from each study unless they represented different subjects. This approach enables one to examine different outcomes while ensuring independence among the findings for each outcome (Gleser & Olkin, 1994). Secondly, multiple effect sizes provided by the same subjects for the same category of outcome (e.g., achievement measured by the post-test and by the delayed post-test) were dealt with by randomly taking a single value from the set of correlated effect sizes per feature for each affected study. This method eliminates the problem of dependency while ensuring that all levels of a study feature were represented (Lou et al., 1996). For example, for the analysis of the "outcome measure time," the selection of within-group findings was made randomly from among outcomes measured by immediate post-tests and delayed post-tests. This method was applied after all the study findings had been extracted and coded. Thirdly, when findings within the same category of outcomes in a study were not distinguishable by any of the study features coded, the effect sizes were averaged.

The study findings were extracted by two coders separately. The initial coding agreement on the number of findings to extract per study was 87.16%. Disagreements between the coders were resolved through subsequent discussion and further review of the disputed findings. Overall, 710 findings were extracted prior to random sampling within studies. After random sampling, 486 independent findings were selected for analysis.

Data Analyses

For each outcome, the unit of analysis was the independent study finding. Data screening was first performed using the SPSS for Windows (SPSS, 1998) frequency and descriptive procedures. Several study features with almost no variability (e.g., measure source, setting of collaboration) or with over 90% missing data (e.g., technical support, content support) were dropped from further analysis. Categories within some variables (e.g., group size, subject areas, and type of learners) were combined based on frequency distributions, conceptual meaning, and the preliminary results from the homogeneity analyses.

Outlier analyses were performed using standardized residual procedures (Hedges & Olkin, 1985). A few outliers with standardized residuals larger than +/-2.00 were identified. These data were then carefully examined to see if there were any computational errors in the studies or if there was any feature in these studies that made them different from other studies. Two computational errors were found in the original source material for one study and their values were corrected based on other information available in the study. For other outliers, no computational or other serious errors were found. In addition, no obvious difference was found between these data and others in terms of their study features. Consequently, it was decided that these data should be included in the data analyses, especially in the study features analyses since excluding them might lead to biased results. However, in order to avoid their over-influence due to their extreme values, these effect sizes were modified (i.e., their absolute values reduced) to bring their residuals just equal to +/-2.00 (Tabachnick & Fidell, 1996).

Effect sizes extracted from studies were then aggregated and tested for homogeneity (Hedges & Olkin, 1985). Each effect size was first corrected for bias and weighted by the inverse of its sampling variance. Thus, more weight was given to findings that were based on larger sample sizes. The weighted effect sizes were then aggregated to form an overall weighted mean estimate of the small group learning effects (d^sub +^). The significance of d^sub +^ was judged by its 95% confidence interval. If the confidence interval did not contain zero, d^sub +^ was considered significantly positive or negative depending on the sign of the mean value. To determine whether the findings shared a common effect size, the set of effect sizes was tested for homogeneity by the homogeneity statistics (QT). When all findings share the same population effect size, Q^sub T^ has an approximate chi-square distribution with k - 1 degrees of freedom, where k is the number of effect sizes. If the obtained Q^sub T^ value is larger than the critical value, the findings are determined to be significantly heterogeneous, meaning that there is more variability in the effect sizes than chance fluctuation would allow.

For two of the significantly heterogeneous outcomes (i.e., individual achievement and group task performance), study features analyses were performed, first univariately and then with multiple regression, to identify factors that significantly moderated the effects of social context on each of the two cognitive outcomes.

Univariate Analyses of Study Features

In the univariate analyses, each study feature was tested through two homogeneity statistics, between-class homogeneity (Q^sub B^) and within-class homogeneity (Q^sub W^). Q^sub B^ tests for homogeneity of effect sizes across classes. It has an approximate chi-square distribution with p - 1 degrees of freedom, where p is the number of classes. If Q^sub B^ is greater than the critical value, it indicates a significant difference among the classes of effect sizes. When a study feature had more than two classes, Scheffe post-hoc comparisons were performed to control for Type I error rate. Q^sub W^ indicates whether the effect sizes within each class are homogeneous. It has an approximate chi-square distribution with m - 1 degrees of freedom, where m is the number of effect sizes in each class. If Q^sub W^ is greater than the critical value, it indicates that the effect sizes within the class are heterogeneous. Univariate study features analyses were conducted using the meta-analysis software DSTAT (Johnson, 1989) for its relative convenience in analyzing a large number of variables.

Multiple Regression Model Testing

Multiple regression models were tested using SPSS for Windows (SPSS, 1998). Based on the results from the univariate analyses, two weighted least squares multiple regression analyses were performed for each outcome. Analysis 1 aimed to identify study features that accounted for significant unique variances in the findings. All the significant predictors identified from the univariate analyses were entered as one block in a simple weighted least squares regression. Significance of each regression coefficient was determined by z test^sup 2^. In Analysis 2, hierarchical weighted least squares regressions (Hedges & Olkin, 1985) were performed to develop a parsimonious model. First, all univariately significant study features were entered in blocks stepwise in this order: grouping characteristics, technology and task characteristics, learner characteristics, and publication status. Next, all other nonsignificant variables were entered stepwise to see if any additional variance might be explained by other variables. At each block, only variables that explained significant additional variance throughout the model testing were retained.

In the weighted least squares multiple regression, the sums of squares for regression (Q^sub R^) (which is similar to Q^sub B^ in the univariate categorical model analysis) has an approximate chi-square distribution with p - 1 degrees of freedom, where p is the number of variables entered. Additional variance explained by each variable is the difference between QR at the current step and at previous step (i.e., Q^sub R^ increment), which is tested as a chi-square with 1 degree of freedom when the variable is dichotomous. Model specification is tested by goodness-of-fit statistics Q^sub E^ (which is similar to Q^sub W^ in the univariate categorical model analysis) with k -p degrees of freedom.

The multiple regression analyses have two advantages over the univariate analyses. First, in the univariate analyses, the Type I error rate may be inflated due to the number of tests that are performed. In the multiple regression analyses, the error rate is controlled. The second advantage of the multiple regression analyses is that they can control for shared variance among the study features to develop a parsimonious model.

All variables were dummy coded into dichotomous variables for the multiple regression analyses. A few variables with more than two levels were combined into dichotomous variables based on the post-hoc analyses results of each of these study features. The higher values) was coded "1", the lower values) was coded "0". The missing data for each variable were coded either "1" or "0" depending on whether the mean effect size of the missing data was similar to the mean effect size for the higher value or the lower value. We chose to compute the fewest dichotomous dummy variables to avoid problems with low statistical power had we created a large number of dummy variables to represent multiple values of each variable (Lou, Abrami, & Spence, 2000; Abrami et al., 2000). The recoding was done globally for the heterogeneous outcomes analyzed with primary consideration given to the achievement outcome and secondly to the pattern that appeared to exist across the outcomes.

Results

In total, 486 independent effect sizes were extracted from 122 studies involving a total of 11,317 learners comparing the effects of small group learning with CT versus individual learning with CT on student individual achievement, group task performance, and several process and affective outcomes. Most of the individual achievement and group task performance outcomes were measured by locally developed or teacher-made instruments or criteria specific to what had been learned on the computer tasks. The majority of the studies were well controlled, employing either random assignment of students to experimental and control conditions or using statistical control for quasi-experimental studies. About half of the studies were published journal articles and half were unpublished reports or doctoral dissertations.

Overall Effects of Social Context on Student Cognitive, Process, and Affective Outcomes

Table 3 presents the number of independent findings extracted, number of studies involved, the weighted mean effect size, 95% confidence interval and overall homogeneity statistics for each of the cognitive, process, and affective outcomes analyzed.

The overall effect of social context on individual achievement was based on 178 independent effect sizes extracted from 100 studies. The mean weighted effect size (d^sub +^) was +0.16 (95% confidence interval is +0.12 to +0.20; and Q^sub T^ = 341.95, df = 177, p < .05) before outlier procedures. Individual effect sizes ranged from -1.14 to +3.37, with 105 effect sizes above zero favoring learning in groups, 15 effect sizes equal to zero, and 58 effect sizes below zero favoring individual learning. Fifteen outliers with standardized residuals larger than +/-2.00 were identified. After outlier procedures, the mean effect size was +0.15 (95% confidence interval is +0.11 to +0.19). The results indicate that, on average, there was a small but significantly positive effect of small group learning on student achievement as measured by individually administered immediate or delayed post-tests. In general, average students (i.e., those at the 50th percentile) learning in small groups achieved at slightly above average (i.e., at about the 56th percentile) compared to students learning individually. However, homogeneity statistics (Q^sub T^ = 259.55, df= 177, p < .05) indicate that the findings on individual achievement were significantly heterogeneous both before and after the outlier procedure.

Thirty-nine independent effect sizes were extracted from 22 studies that explored the relationship between social context and performance where students learning in groups completed a group task and where students working individually completed an individual task. Group task performance measures included number of words or letters correct, number of problems, cases or puzzles solved, degree of success, percentage of correct responses, number of errors made (with the positive or negative sign of the effect size reversed), number of questions correct, quality of drawing, writing, projects or simulation results, number of errors identified or corrected, and scores on group assignments. The mean weighted effect size was +031, which was significantly different from zero (95% confidence interval is +0.20 to +0.43). The results indicate that, on average, there was a moderate positive effect of small group learning on group task performance. In general, groups performed significantly better than individuals during the study. However, the variability in the findings suggested significant heterogeneity (Q^sub T^ = 102.90, df = 38, p < .05). The effect sizes ranged from -0.86 to +2.53, with 30 effect sizes above zero favoring group task performance and 9 effect sizes below zero favoring individual task performance.

Enlarge 200% Enlarge 400%

Relatively fewer studies reported learning processes and student task behaviors. Based on the findings extracted and analyzed in this review, small group learning had significantly positive effects on several learning processes. On average, students learning in groups had a significantly higher frequency of positive peer interaction (d^sub +^ = +0.33), a higher frequency of using appropriate learning or task strategies (d^sub +^ = +0.50), were more perseverant on tasks (d^sub +^ = +0.48), and more students succeeded (d^sub +^ = +0.28) than those learning individually. Students learning individually on average interacted more with computer programs (d^sub +^ = -0.19), requested significantly more help from the teacher or monitor (d^sub +^ = -0.67) and accomplished tasks faster than those working in groups (d^sub +^ = -0.16). No significant differences were found between groups and individuals on amount of tasks attempted. Homogeneity statistics indicate that the findings on perseverance (Q^sub T^ = 2.60, df = 3), success rate (Q^sub T^ = 4.28, df = 5), and interactivity with programs (Q^sub T^ = 28.22, df = 16) were homogeneous, suggesting that the effect sizes were consistent. However, each set of effect sizes for the other measures was significantly heterogeneous, indicating considerable variability in the findings within each of these process measures.

Results on affective outcomes indicate that working with others in small groups when learning with CT had significantly positive effects on student attitude toward group work (d^sub +^ = +0.52), and attitude toward classmates (d^sub +^ = +0.29). No significant differences were found between students learning in small groups or individually on their attitudes toward computers, subject or instruction, or academic self-concept. Homogeneity statistics indicate that the findings on student attitude toward classmates (Q^sub T^ = 7.86, df= 10), computers (Q^sub T^= 21.23, df = 26), and academic self-concept (Q^sub T^ = 1.78, df = 9) were homogeneous, suggesting that the effect sizes were consistent. However, findings on student attitude toward group work and toward learning with computers were significantly heterogeneous, indicating considerable variability in the findings within each of the two datasets.

In order to identify any potential pedagogical and/or contextual factors that may moderate the effects of social context, study features analyses were performed on each of the two heterogeneous cognitive outcomes.

What study features moderate the effects of social context on individual achievement in learning with CT? And what are the optimal conditions for small group learning?

Twenty-three study features were analyzed to identify factors that significantly moderated the effects of social context on individual achievement. Several study features (including outcome measure source, design orientation, teacher support, setting of collaboration, presence of others, and amount of peer interaction) were dropped from the analyses due to almost no variability or missing values in 90% of the findings.

Table 4 presents the results of the univariate analyses. Of the 23 study features analyzed, 9 study features were significantly related to the variability in the individual achievement findings. Each of the significant study features is described below.

Publication status. Effects of social context on student individual achievement were significantly more positive (Q^sub B^ = 5.11, df = 1, p < .05) in published journal articles (d^sub +^ = +0.20) than in unpublished conference reports and dissertations (d^sub +^ = +0.10). However, both means were significantly positive favoring student learning in small groups.

Types of programs. The types of programs with which students were learning was significantly related to the effects of social context on student individual achievement (Q^sub B^ = 13.07, df = 2, p < .05). Five types of computer programs were initially identified and coded. They were: tutorial, drill-and-practice, exploratory environments, productivity tools, and programming languages. Based on both conceptual similarity and post hoc analyses, tutorial and drill-andpractice were combined as tutor; exploratory environments and productivity tools were combined as exploratory/tool. Effect sizes were significantly larger when students were learning with tutor programs (d^sub +^ = +0.20) or programming languages (d^sub +^ = +0.22) than when using exploratory or tool programs (d^sub +^ = +0.04). While the former two means were significantly positive, the latter was not significantly different from zero.

Subject. Effects of social context on student individual achievement varied in different subject areas (Q^sub B^ = 7.95, df = 2, p < .05). Initially, subjects were coded into six categories: math, science, reading/writing and language arts, computer skills, social studies, and other. Due to the small sample size in reading/writing and language arts and no significant differences among the mean effect sizes for math, science, and reading/writing and language arts, these three categories were combined; similarly, social studies and other were combined as the mean effect sizes for the two categories were not significantly different from each other. Analysis of the resulting three categories indicate that the effects of social context on student individual achievement were larger when the subjects involved were computer skills (d^sub +^ = +0.24), social sciences and other (d^sub +^ = +0.20) than when the subjects were math/science/language arts (d^sub +^ = +0.11). However, all three mean effect sizes were significantly positive favoring small group learning with CT over individual learning with CT.

Task structure. Effects of social context on student individual achievement were significantly larger (Q^sub B^ = 5.64, df = 1, p < .05) for closed-ended tasks (d^sub +^ = +0.22) than for open-ended tasks (d^sub +^ = +0.11). Still, both means were significantly positive, indicating the superiority of small group learning with CT over individual learning with CT for both types of task structure.

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Group composition. Type of group composition was significantly related to the effects of social context on student individual achievement (Q^sub B^ = 9.69, df = 4, p < .05). The effect sizes were significantly positive for both heterogeneous ability groups (d^sub +^ = +0.21) and homogeneous ability groups (d^sub +^ = +0.22). That is, students using CT in either homogeneous or heterogeneous ability groups outperformed students working alone using CT when all students were measured on individual tests of achievement. When groups were formed based on either homogeneous gender (d^sub +^ = -0.04) or heterogeneous gender (d^sub +^ =-0.07) the effects of group composition did not differ significantly from zero. Finally, the mean effect size for groups based on mixed criteria (d^sub +^ = +0.13) was also positive but not significantly different from zero.

Group learning strategy. The group learning strategy employed was significantly related to the effects of social context on student individual achievement (Q^sub B^ = 16.11, df = 2, p < .05). Effect sizes were significantly more positive when specific cooperative learning strategies were employed (d^sub +^ = +0.21) than when students were generally encouraged to work together (d^sub +^ = -0.04) or when students in groups worked under individualistic goals or when no group learning strategy was described in the study (d^sub +^ = +0.08), with the latter two means not significantly different from zero.

Group work experience or instruction. Effects of social context on student individual achievement were significantly more positive (Q^sub B^ = 16.24, df = 1, p < .05) when students had group work experience or instruction (d^sub +^ = +0.29) than when no such information was reported (d^sub +^ = +0.10). Both were significantly positive when compared to students learning with CT alone.

Group size. Effects of social context on student individual achievement were significantly more positive (Q^sub B^ = 5.05, df = 1, p < .05) when students worked in pairs (d+ = +0.18) than when they worked in three to five member groups (d^sub +^ = +0.08). Both group size conditions were significantly positive compared to students learning alone with CT.

Relative ability level of students. Effects of social context on student individual achievement were significantly related to the relative ability level of the students (QB = 12.09, df = 3, p < .05). There was a moderate positive effect of social context for low ability learners (d+ = +0.34) and a small positive effect for high ability students (d+ = +0.24). For medium ability learners, the effects were also positive but not significantly different from zero (d+ = +0.09). Effect sizes for low ability students were significantly larger than those for medium ability students.

Other features. Most of the studies were published in the 1990s. The findings from the studies published in the last five years were not significantly different from those published in the earlier years. Over 90% of the studies were well controlled. The results from a few studies that did not use experimental control were not significantly different from the others. Type of feedback, types of tasks, task familiarity, task difficulty, number of sessions, session duration, grade level, gender, computer experience, instructional control, and whether achievement outcomes measured were of higher-order skills or lower-order skills were not found to be significantly related to the variability in the effects of social context on student individual achievement.

The next phase of the analysis of individual student achievement used multiple regression as a tool for model development. Analysis 1 identified unique variance explained. Analysis 2 identified a parsimonious model of important predictors.

Multiple Regression Analysis 1: Testing for unique variances using univariately significant predictors. The nine significant predictors (p <.05) identified from the univariate study features analyses were tested for their unique variances in a weighted least squares multiple regression. All variables were entered as one block. Of the nine variables entered, four accounted for significant unique variances in the findings: publication status (4.72%), group work experience/instruction (3.83%), subject (3.21%), and relative ability level (2.00%). Another 8.36% of the systematic variance was shared by the nine variables entered. Overall, the nine study features accounted for 22.12% of the total variance. Goodness-of-fit statistics WE = 197.44, df = 167) indicate that the remaining variance can be explained by sampling error.

Multiple Regression Analysis 2: Hierarchical regression model development. Results of the hierarchical regression analyses are presented in Table 5. Six variables entered the model. Group work experience/instruction, subject, relative ability level, and publication status that were significant in Analysis 1 remained significant in the hierarchical regression model. Two univariately significant variables, group learning strategy and type of program, that were not significant in Analysis 1 each accounted for a significant amount of variance in the hierachical regression model. Together, the six variables accounted for 21.12% of the total variance in the findings. Goodness-of-fit statistics (Q^sub E^ = 199.96, df = 170) indicates that the model fits the data and that the remaining variance may be explained by sampling error. Three other study features including task structure, group composition, and group size were significant when analyzed separately but were not significant in the multiple regression model due to their correlation with other predictors.

Table 6 presents the regression coefficients and their standard errors in the optimal regression model. The results indicate that the effects of small group learning with CT on individual achievement were significantly larger when: (a) students had group work experience or specific instruction for group work rather than when no such experience or instruction was reported; (b) cooperative group learning strategies were employed rather than general encouragement only or individual learning strategies were employed; (c) programs involved tutorials or practice or programming languages rather than exploratory environments or as tools for other tasks; (d) subjects involved social sciences or computer skills rather than mathematics, science, reading, and language arts; (e) students were relatively low in ability rather than medium or high in ability; and (f) studies were published in journals rather than not published. When all the positive conditions were present, students learning in small groups could achieve 0.66 standard deviation more than those learning individually. When none of the positive conditions were present, students learning individually could learn 0.20 standard deviation more than those learning in groups.

What study features moderate the effects of social context when students learn with CT on group task performance? And what are the conditions for optimal group task performance?

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Seventeen study features were analyzed to explore the variability in the group task performance data. In addition to those that were dropped from analysis on individual achievement, a few more study features were dropped from analysis on group task performance due to almost no variability or missing values in 90% or more of the findings. These included group work experience or instruction, session duration, relative ability level, and computer experience. In most of these studies, experimental sessions lasted from about 10 to 60 minutes; there was no description about group work experience or instruction; no description about computer experience; and no separate results for students of different relative ability levels. Outcome measure time does not apply here since all group task performance outcomes were measured during the study. Outcome type also does not apply since only task performance was measured here and the difference in task type is already represented by another study feature (i.e., type of task).

Table 7 presents the results of the univariate study features analyses on the group task performance data. Of the 17 study features analyzed, 5 study features were significantly related to the effects of social context on group task performance. Each of the significant study features is described below.

Feedback. Type of feedback provided by computer programs was significantly related to the effects of social context on group task performance, (Q^sub B^ = 16.62, df = 2, p < .05). Effect sizes were significantly more positive when programs provided no feedback (d^sub +^ = +0.47) or minimal feedback (d^sub +^ = +0.29) than when elaborate feedback was available in the computer programs (d^sub +^ = -0.24). While the former was significantly positive favoring groups, the latter was significantly negative favoring individuals. Individuals benefit from computer-based feedback but groups do better without computer-based feedback when completing group tasks.

Instructional control. Effect sizes on group task performance were significantly more positive (Q^sub B^ = 9.68, dF = 1, p < .05) when the software was mostly learner-controlled (d^sub +^ = +0.41) than when the software was mostly systemcontrolled (d^sub +^ = -0.02). While the former mean effect size was significantly positive, the latter was not different from zero. The advantage of working together and completing a group task was enhanced when students working together had control over the software they were using. This advantage disappeared when students working together on a group task had no control over the software they were using. Task difficulty. Level of task difficulty was significantly related to the effects of social context on group task performance (Q^sub B^ = 8.89, df = 2, p < .05). Significantly more positive effect sizes were found when tasks were difficult (d^sub +^ = +0.13) than when tasks were moderately difficult (d^sub +^ = -0.34) or not difficult (d^sub +^ = -0.57). When tasks were not difficult, the mean effect size was significantly negative favoring individuals (d^sub +^ = -0.57); when tasks were moderately difficult, the mean effect size was also negative (d^sub +^ = -0.34); but when tasks were difficult, the mean effect size was more positive favoring students working in groups (d^sub +^ = +0.13). However, the latter two means were not statistically different from zero.

Group composition. Effect sizes on group task performance varied significantly for different group compositions (Q^sub B^ = 27.03, df= 4, p < .05). When groups were formed based on mixed criteria (i.e., ability and other criteria), the effect size was large (d^sub +^ = +1. 15) and significant. When groups were homogeneous in terms of gender, effect sizes were moderately large (d^sub +^ = +0.51) and also significant. Finally, the mean effect size (d^sub +^ = +0.29) for homogeneous ability groups was also significantly positive. However, the mean effect sizes for heterogeneous ability groups and heterogeneous gender were not significantly different from zero. Not all groups are created equal: Working in groups on a group task is superior to working alone on an individual task when groups are composed using mixed criteria, when groups are homogeneous in ability, or when groups are either all males or all females.

Group size. Effect sizes on group task performance were significantly larger (Q^sub B^ = 15.34, df = 1, p < .05) for three- to five-member groups (d^sub +^ = +0. 87) than for pairs (d^sub +^ = +0.22), although both means were significantly positive. Working in larger groups and completing group tasks is generally superior to working in smaller groups.

Other features. Student equivalence across conditions, publication status, publication year, type of program, subject, type of task, task structure, task familiarity, group learning strategy, number of sessions, grade level, and gender were not found to be significantly related to the variability in the effects of social context on group task performance.

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

The next phase of the analysis of group task performance used multiple regression as a tool for model development. Analysis I identified unique variance explained. Analysis 2 identified a parsimonious model of important predictors.

Multiple Regression Analysis 1: Testing for unique variances using univariately significant predictors. Unique variances accounted for by each variable were tested in a weighted least squares multiple regression with all five significant study features identified from the univariate analyses entered in one block. Three study features were significant, each accounting for a significant amount of unique variance in the findings: task difficulty (8.96%), feedback (5.53%), and group size (5.62%). Another 27.77% of the systematic variance was shared by the 5 variables entered. Overall, the five study features accounted for 47.88% of the total variance. Goodness-of-fit statistics (Q^sub E^ = 56.22, df = 33), however, indicated that the model does not fit the data and that there may be other significant predictors which were not included in this model.

Multiple Regression Analysis 2: Hierarchical regression model development. Results of these analyses are presented in Table 8. Group size, task difficulty, and feedback that were significant in Analysis 1 remained significant in the hierarchical regression. After variance due to the three variables had been accounted for, task structure, which was not significant in the univariate analysis, accounted for a significant amount of additional variance (Q^sub R increment= 16.93). Together, the four variables accounted for 60.81% of the total variability. Goodness-of-fit statistics (Q^sub E^ = 42.27, df = 34) indicate that the model fits the data and that the remaining variability may be explained by sampling error. Two other study features, instructional control and group composition, that were significant when analyzed separately were not significant in the multiple regression analyses due to their correlation with other predictors.

Table 9 presents the regression coefficients and their standard errors in the optimal regression model. The results indicate that the superiority of group performance over individual performance was stronger when: (a) group size was relatively large with three to five members; (b) the learning tasks were difficult; (c) programs provided minimal or no feedback, and (d) the tasks' structure was closed-ended. When all the positive conditions were present, group performance was about 3.02 standard deviation better than individual performance. When none of the positive conditions were present, individual performance would be about 1.66 standard deviation better than group performance. However, the finding concerning task structure may not be stable since it was not a significant predictor when analyzed separately, where the mean effect sizes for open-ended tasks and closed-ended tasks were both significantly positive favoring group task performance over individual task performance.

Discussion

Based on a total of 486 independent findings extracted from 122 studies involving 11,317 learners, the results of the series of meta-analyses conducted in this review indicate that social context plays an important role when students learn with CT. In general, small group learning with CT had more favorable effects than individual learning with CT on student cognitive, process and affective outcomes. On average, there was a small but significantly positive effect of social context on student individual achievement (mean ES = +0.15) and a moderate positive effect on group task performance (mean ES = +0.31). These positive results indicate that when working with CT in small groups, students in general produced substantially better group products than individual products and they also gained more individual knowledge than those learning with CT individually.

Analyses of several learning processes indicate that students learning with CT in small groups or individually tended to exhibit different task behaviors. Students learning individually with CT often accomplished tasks faster (mean ES = -0. 16) through interacting more with the programs (mean ES = -0. 19) and by getting more help from the teacher (mean ES = -0.67). In contrast, students learning in small groups benefited from greater social and cognitive interaction with peers (mean ES = +033), increased use of appropriate learning strategies (mean ES = +0.50), and better task perseverance (mean ES = +0.48). Finally, small group learning with CT had a significant positive effect on student attitudes toward group work (mean ES = +0.52) and toward classmates (mean ES = +0.29).

Enlarge 200% Enlarge 400%

However, not all groups perform equally well and not all students learning in small groups using CT learned more than those learning individually with CT under all conditions. Through weighted least squares univariate and multiple regression analyses of individual achievement and group task performance outcomes, we found that the significant variability in each of the two cognitive outcomes could be accounted for by a few technology, task, grouping and learner characteristics.

Pedagogical and Contextual Factors that Moderate the Effects of Small Group Learning with CT on Student Individual Achievement

The study features that accounted for the most variability in the individual achievement findings were: group work experience or instruction, group learning strategies, type of program, subject, relative ability level, and publication status, with each accounting for a significant amount of independent variance. Group size was also a significant predictor when analyzed separately but not in the multiple regression analyses due to its correlation with other predictors. The effects of small group learning were significantly enhanced when: (a) students had group work experience or instruction; (b) specific cooperative learning strategies were employed; (c) group size was small (i.e., two members); (d) using tutorials or practice software or programming languages; (e) learning computer skills, social sciences and other subjects such as management and social studies; and (f ) students were either relatively low in ability or relatively high in ability. When all the positive conditions are present, especially when studies were published in journals, moderate positive effects of social context (mean ES = +0.66) may be expected.

We did not find any category within a study feature, when analyzed separately, that showed significant negative effects of social context favoring individual learning on individual achievement. A few conditions were not significant univariately. These included conditions in which: (a) no specific cooperative learning strategies were used to facilitate group learning; (b) programs involved exploratory environments or were used as tools for other tasks; and (c) students were relatively medium in ability. Collectively, when all these conditions are present, especially when the subject matter involves mathematics, science, or language arts and the studies were reported in unpublished conference papers and dissertations, a small negative effect of social context (mean ES = -0.22) favoring individual learning with CT may be expected.

These results suggest that prior group learning experience and the teacher's use of cooperative learning strategies are important pedagogical factors that may influence how much students learn when working in small groups using CT. Explanations of group dynamics suggest that not all groups function well; for example, groups often do not function well when some members exert only minimal effort (Sharan & Sharan, 1976, 1992; Shepperd, 1993). Students need practice working together on group activities and training in how to work collaboratively (Webb, 1997; Farivar & Webb, 1994a; 1994b). Experience in group work may enable members to use acquired strategies for effective group work. Specific instruction for cooperative learning ensures that students learning in small groups will have positive interdependence as well as individual accountability that are essential qualities of effective cooperative learning (Abrami et al., 1995).

The more positive effects of small group learning with CT when specific cooperative learning strategies were employed are consistent with the meta-analysis by Lou et al. (1996 and Lou, Abrami, & Spence, 2000) of within-class grouping and with the quantitative syntheses of the cooperative learning literature (Johnson & Johnson, 1989; Slavin, 1989). Abrami et al. (1995) summarized myriad motivational and learning explanations of the positive effects of cooperative learning. These explanations may help illuminate the positive effects of social context when students learn with CT.

Motivational explanations concentrate on explaining student interest in, involvement with, and persistence at learning. Slavin (1992) argued that both cooperative incentives and cooperative task structures increase performance when they lead to encouragement among group members to perform the group task and to help one another in doing so. Johnson and Johnson (1994) used the theory of social interdependence to explain how the perception of interdependence among students motivates them to engage in promotive interactions that facilitate the realization of mutual goals. Ames (1984) suggested that a morality-based motivational system underlies cooperative goal structures such that students are motivated by the desire to help others and place special emphasis on individual and group efforts to achieve, making causal ascriptions to effort more salient than attributions to ability. Social cohesion explanations (Cohen, 1994; Sharan & Sharan, 1992) argue for the pre-eminent role of group cohesion which arises from care and concern for the group and its members.

Learning explanations concentrate on how the interactions among students affect their understanding and cognitive processes. Cognitive elaboration perspectives (Dansereau, 1985; Webb, 1989) suggest that the learner must engage in cognitive restructuring if information is to be retained and related to information already in memory, particularly by giving and receiving elaborated explanations. Johnson and Johnson (1992) describe several ways that the promotive interactions affect student thinking including: oral rehearsal, perspective-taking, peer monitoring, feedback, and cognitive controversy. Damon (1984) highlighted the cognitivedevelopmental perspectives of Piaget and Vygotsky who both emphasized how the interaction among students around cognitively appropriate tasks increases the mastery of critical concepts via discovery, idea generation, argumentation, verification, and criticism. Other explanations focus on practice effects, time-on-task, and classroom organization explanations.

The differential effects of small group learning for students of different relative ability levels are consistent with those found in Lou et al. (1996). The heterogeneous effects on individual achievement occurred mainly in the heterogeneous ability groups. Lou et al. (1996), Webb (1997), and Webb & Palincsar (1996) explained that in heterogeneous ability groups, low and high ability students benefit from receiving and giving explanations. For example, receiving explanations may help low ability students correct misconceptions and acquire appropriate learning strategies. Giving explanations may help high ability students clarify and organize their own learning. In contrast, medium ability students may benefit less from learning in small heterogeneous ability groups as they may neither give explanations as frequently as high ability students nor receive explanations as frequently as low ability students.

When using CT, students learned more working in pairs than in three to fivemember groups. This finding is different from the within-class grouping research (Lou et al., 1996) where the optimal group size was larger. The difference may be due to the physical constraints associated with computer use. Group size may have to be small enough for all group members to sit comfortably around the computer in face-to-face collaborations in order to participate equally and actively. Alternatively, the computer itself may function as a prominent group member or tutor (Crook, 1991), requiring extraordinary coordination among students to insure proper engagement, pace, task sequencing, perspective-taking, and so on.

The effects of social context were more positive with drill-and-practice or tutorial programs than with exploratory or tool programs. There are several plausible explanations for these unexpected findings. First, when working in groups, especially when programs were exploratory in nature, the collaborators may have focused on actions and results rather than taking the time to articulate their mental processes or provide explanations for their actions (Daiute, 1989). Second, motivation may be another plausible explanation. When working with tutorials or drill-- and-practice programs, students may find it more enjoyable and motivating to learn with peers than to work alone. Third, incidental learning outcomes of exploratory programs may not be captured by achievement post-tests, thus under-representing the effects of collaboration.

Factors Moderating the Effects of Social Context on Group Task Perfonnance

The study features that accounted for the most variability in group task performance findings include group size, task difficulty and type of feedback. The superiority of group performance over individual performance was more pronounced when: (a) tasks were especially difficult; (b) groups consisted of three to five members; and (c) no or minimal feedback was available from the programs. When all the optimal conditions are present, a large positive effect of social context of more than 2 standard deviations may be expected on group task performance, as compared to individual task performance.

Work on socially shared cognition and distributed learning (Resnick, Levine, & Teasley, 1991; Salomon, 1993) emphasizes the impact of the social context on learners-both as individuals and within groups in face-to-face as well as computermediated environments-and gives rise to the conceptualization of groups as information processors (Hinsz, Tindale, & Vollrath, 1997). When working together, the group is capable of doing more than any single member by comparing alternative interpretations and solutions, correcting each other's misconceptions, forming a more holistic picture of the problem if the task is complex, or simply pooling resources. This advantage may be especially important when tasks are difficult and when minimal or no feedback is available from the programs. Under these condilions, students working alone may not have all the necessary cognitive resources and skills to complete the tasks well. In addition, when the software is capable of providing elaborate feedback, it may serve as an intellectual partner (Crook, 1991), ameliorating the effect of individual learning.

Differences between Student Individual Achievement and Group Task Performance

The results on student individual achievement and group task performance suggest that the two cognitive outcomes appeared different not only in their mean effect sizes but also in the factors that accounted for the variability in the two outcomes. A comparison of several predictors of individual achievement and group task performance indicated a differential pattern of moderating effects (see Figure 1). When cooperative learning strategies were employed, when students worked in pairs, and when programs involved tutorials or practices or programming languages, there was a small but significantly positive effect on both group task performance and individual achievement. However, when no specific cooperative learning strategies were employed, when students worked in larger three to five member groups, and especially when programs were used as exploratory environments or as tools for other tasks, although there were larger positive effects on group task performance, there were no significant positive effects on individual achievement.

In contrast, the effect sizes involving feedback showed a different pattern of moderating effects across group task performance and individual achievement outcomes. When programs provided minimal or no feedback, positive effects were found on both group task performance and individual achievement. However, when programs provided elaborate feedback, although the mean effect size on group task performance was significantly negative favoring individual learning, a significant positive mean effect size was observed on individual achievement.

These findings suggest that significantly higher group task performance does not necessarily mean significant individual learning, or vice versa. One plausible explanation for these differential effects is the different requirements for group task performance and individual achievement. While the former may reflect the collective wisdom and efforts of all or some of the participating members, the latter requires that each member of the group be actively engaged, interact and learn from each other in order to gain more knowledge from learning together (Webb, 1997). Caution should therefore be exercised when no specific cooperative strategies are used and when group size is larger than two members and especially when programs involve exploratory learning or are used as tools. Under these conditions, although one may generally expect significantly higher group performance over individual task performance, each individual student may not learn equally well.

On the other hand, the differential influence of elaborate feedback on group task performance and individual achievement suggest that articulation of ideas and discussion may be more important in facilitating student learning than simply reading the feedback provided on the computer screens. The cognitive elaboration (Vygotsky, 1978), cognitive dissonance (Piaget, 1954), and peer help and explanation (Webb, 1982a, 1982b) when working with others may create a deeper processing of ideas and, hence, better learning.

These results suggest that group task performance using CT is not the same as individual achievement using CT given the differences in moderating influences. When students work together on group projects, it is important to differentiate group products and individual learning outcomes. There are situations when collaborative task completion is defensible scholastically, demonstrating what a collection is capable of, enhancing motivation and group cohesiveness via pride in a collective accomplishment, and so on. However, if the focus is on individual achievement, effective cooperative learning strategies such as positive interdependence and individual accountability (e.g., requiring students to take turns and agree on answers, to summarize and explain their group's work), emphasizing that all members learn, should be employed to ensure the successful learning of all students.

Enlarge 200% Enlarge 400%

Strengths, Limitations, and Future Directions

This meta-analysis extends knowledge of the role of social context when students learn with CT on various cognitive, process, and affective outcomes. It has addressed the question of whether and to what extent small group learning with CT is more effective than individual learning with CT and on which outcomes. It has identified a number of study features that moderated the effects of social context when learning with CT on group task performance and individual achievement. Through weighted least squares multiple regression analyses, parsimonious models were developed that accounted for the variability of social context effects on group task performance and individual achievement outcomes.

We caution the readers, however, that this meta-analysis, like others, has several limitations. First, meta-analysis results, especially those concerning explanatory features are correlational in nature and, therefore, strong causal inferences are not warranted. Second, as meta-analysts do not have experimental control over data, some of the study features examined had small sample sizes, or missing data, which reduces the sensitivity of the analyses. Third, multiple regression analyses are sensitive to the order variables are entered. Although care was taken to limit the influence from this artifact by testing two models in a different way, we do not claim that the hierarchical regression model is final and conclusive. It is also possible that some other factors not included in primary studies and in this review may provide some additional explanation. Finally, results of this meta-analysis may be limited by the design quality of the programs used in the primary studies. The majority of the programs were designed with an individual orientation or with no special design for group work. The few programs that provided special design for group use such as dual keyboards or computer allocation of turn-taking were of limited success. More effective program designs for small group learning should be developed and tested. For example, a program that is designed for small group use may provide built-in opportunities for each member to articulate and compare choice of task solutions and rationales.

As computers become ubiquitous tools for learning and instruction, and as teachers and students develop greater facility with their use to promote learning, we may learn more about the empowering effects of social context. For now, we are satisfied that old fears of social isolation can be overcome and that students collectively can learn well with technology.

Notes

1An earlier version of the meta-analysis based on fewer studies (1965-1995) was presented at the Annual Meeting of the American Educational Research Association, San Diego, April, 1998 (Lou, Abrami, & Muni, 1998) and in Lou (1999).

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

Enlarge 200% Enlarge 400%

2 The standard error (SE B) in the output of SPSS was adjusted by a factor of the square root of the Mean Square error (MSE) for the regression model according to Hedges and Olkin (1985), because the output in the SPSS was based on a slightly different model than the fixed model used here.

Acknowledgment

This research was supported by a fellowship grant from Fonds pour la formation de chercheurs et l'aide A la Recherche (Government of Quebec) and internal funding from Lousiania State University to the first author and grants from the Social Sciences and Humanities Research Council (Government of Canada) and Fonds pour la formation de chercheurs et l'aide A la recherche (Government of Quebec) to the second author. The authors express their appreciation to Professor Anne Wade for conducting the comprehensive electronic searches used to identify the studies included in the review, several research assistants for collecting and coding the studies with the first author, and several colleagues for reading and making valuable suggestions on earlier versions of the paper.