Applying constructivism: A test for the learner as scientist.

Educational Technology Research & Development, 47 (3),15-31.

By Tom Cobb, Dép. de linguistique et de didactique des langues, Université du Québec à Montréal, Canada.

To appear autumn 1999. 
WWW-prepublication. 
Comments welcome: cobb.tom@uqam.ca.

ABSTRACT

Constructivist learning theory predicts that knowledge encoded from data by learners themselves will be more flexible, transferable, and useful than knowledge encoded for them by experts and transmitted to them by an instructor or other delivery agent. If this prediction is correct, then learners should be modeled as scientists and use the reasoning and technologies of scientists to construct their own knowledge. However, it cannot be taken for granted that the prediction is correct, or correct in every knowledge domain. The present study attempts to establish conditions in which the prediction can be operationalized and tested. It reports on the adaptation of constructivist principles to instructional design in a particular domain, second language vocabulary acquisition. Students learning English for academic purposes in the Sultanate of Oman followed one of two approaches to vocabulary expansion, learning pre-encoded dictionary definitions of words, or constructing definitions for themselves using an adapted version of the computational tools of lexicographers. After 12 weeks, both groups were equal in definitional knowledge of target words, but lexicography group students were more able to transfer their word knowledge to novel contexts.

Le constructivisme appliqué: l'apprenant devient chercheur. La théorie d'apprentissage constructiviste prédit que un savoir qui est encodé par les apprenants eux-mêmes à partir de données brutes sera plus maniable et transférable qu'un savoir préencodé qui est simplement donné aux apprenants. Alors que cette prédiction plaît beaucoup au plan intuitif, au plan empirique elle a été peu mise à l'épreuve. L'étude de Miller et Gildea (1985) constitue une exception à cette tendance générale. Elle démontre que l'acquisition des mots de vocabulaire peut se faire plus facilement par le biais des phrases (savoir construit par les apprenants) que par le biais des définitions (savoir construit pour les apprenants). La présente étude part du travail de Miller et Gildea pour monter un programme constructiviste qui cherche à faciliter l'acquisition des mots de vocabulaire dans langue seconde au niveau universitaire. Dans le cadre de l'étude, des étudiants en anglais langue seconde ont suivi l'un ou l'autre de deux programmes visant l'expansion de leur vocabulaire. Ceux qui ont suivi le programme expérimental ont contruit leurs propres définitions de nouveaux mots en se servant des mêmes outils que des lexicologues professionnels. Ceux qui ont suivi le programme de contrôle ont cherché les mêmes mots dans un dictionnaire. Au bout de 12 semaines, les deux groupes étaient au même niveau en ce qui concernait leur capacité de définir les mots en question, mais les étudiants suivaient le programme expérimental se distinguaient de façon significative en ce qui concernait leur capacité de retenir, de réutiliser et de tranferer leur savoir à de nouveaux contextes. Nous discuterons également du rôle d'une approche constructiviste dans la structuration de l'enseignement et l'utilisation des médias.

INTRODUCTION

A constructivist model of learning has been proposed as an alternative to the transmission model implicit in all behaviorist and some cognitive approaches (Brown, Collins, & Duguid, 1989). There are a number of competing constructivisms (Phillips, 1995), but the underlying principle is that two kinds of knowledge are created by two kinds of learning. One kind is inert, easily forgotten, and untransferable beyond its initial context of learning because it was "pre-emptively encoded" (Spiro, Coulson, Feltovich, & Anderson, 1988, p. 377) by an expert for transmission to a learner. The other kind is memorable and transferable to novel contexts because learners have encoded it for themselves out of raw data, or at least raised it from a lower to a higher level of organization, by forming and testing hypotheses as professional scientists do (Resnick, 1987).

Constructivism has recently become interesting to educational technologists, partly because of the ways that information technology is impacting on life, learning, and work (Duffy & Jonassen, 1991), and partly because it offers a new approach to instructional design as interest declines in the instructional systems technology model (Bednar, Cunningham, Duffy & Perry, 1991). However, up to now a role for constructivism has been discussed more in principle than in practice, and claims about the kinds of knowledge it produces remain largely untested.
 
 

CONSTRUCTIVISM - WHAT'S NEW?

Constructivism is reminiscent of the discovery approaches to learning that have surfaced periodically since Dewey (1938), whereby learners learn best what they discover or can be led to discover for themselves. In any precise epistemology, the discovery of (old) knowledge is not identical to the construction of (new) knowledge, but the approaches are similar in terms of benefits claimed and challenges posed. For Brown and Campione (1994, p. 229) discovery learning is now "fully incorporated" into constructive learning. The approaches share at least one central notion, that learners should attempt to resemble scientists in non-trivial ways, engaging in independent theory formation and hypothesis testing.

While anyone can think of cases where knowledge is not ideally gained through hypothesis testing (e.g., the knowledge that children should not play with matches), with a few such provisos the learner-as-scientist idea has perennial appeal. So, is constructivism destined to join discovery learning in the long list of educational enthusiasms that come and go, never articulated clearly enough to be tested, or in vogue long enough to prove their theoretical interest or practical worth? For a number of reasons, constructivism may be more than just another swing of the discovery pendulum.

The common interest in modeling learners as scientists masks a number of differences in the models themselves.

First, earlier discussions of the learner-as-scientist were often conducted in vague terms, as noted by Ausubel (1968) and Bereiter (1994). In retrospect, modeling learners as scientists was of limited value when it was not well understood what scientists do or how they think. Now, descriptions are available of scientists' and other experts' information-processing and knowledge-constructing procedures (e.g., Chi, Glaser & Farr, 1988), to the point that expert models may be detailed enough for adaptation as learner models.

Second, discovery learning was an instructional methodology mainly of interest in educational circles, while modern constructivism is a theory of mind and brain informing research in several areas including neuroscience (Quartz & Sejnowski, 1997).

Third, simplistic notions of "better" knowledge deriving in principle from data-driven learning, while still present, do not dominate the present discussion of constructivism. A typical current view is that information in various states of organization from raw data to fully formed representations can be successfully integrated into human memory, and the level that is better in a particular case depends on what input is available and what use the output or knowledge will serve (Anderson, Reder & Simon, 1995, 1996; Merrill, 1991). Indeed, the perils of learners' constructing their own knowledge now seem at least as apparent as the advantages, following the many studies of the misconceptions to which naive theory construction is prone (e.g., McClosky, 1983), and accordingly many constructivists now take for granted some form of scaffolded (Lajoie & Lesgold, 1989) or guided (Brown & Campione, 1994) approach.

In other words, a sounder and more sober learner-as-scientist theory may now be available for instructional purposes, assuming there is any reason to remain interested in it. One reason may be that some version of constructive learning is now upon us whether we like it or not. Duffy and Jonassen (1991; 1992) argue that a redefinition of learning and intelligence is taking place under the pressure of the information revolution. Until recently, an education provided a store of tried and true, preconstructed knowledge for use with minor modifications over a lifetime, but now an education must impart the ability to compute useful new knowledge on the fly on a just-in-time basis from an unending flow of novel, mixed-quality information. The goals of education should be extended to impart this ability, and constructivist approaches to instructional design may offer a framework for doing this.
 
 

CONSTRUCTIVISM AND EDUCATIONAL TECHNOLOGY.

Duffy and Jonassen's (1991) piece is written from the educational technology perspective, and indeed the task of devising instructional paradigms for the information age seems a natural one for our field. Two of our traditional strengths address two of the challenges of implementing constructivist learning in practical settings. (a) One strength is our experience in designing independent learning environments, from teaching machines to interactive tutors. If learners are to grapple with raw data in their own way and time in any real sense, then some type of independent learning environment will be necessary. (b) Another strength is our techniques for knowledge engineering, or quizzing experts for what they know and how they think in order to communicate this information to others. If learners are to be modeled as experts in any real sense, then detailed accounts of what many types of experts know and do will be necessary.

Not surprisingly, many educational technologists have shown enthusiasm for constructivism, some even likening their discovery of it to a "conversion" (Bednar et al., 1991, p. 91). Bednar and her colleagues argue that constructivism highlights what was always best in the educational technology approach and can serve as its new theoretical center, providing an alternative to both the waning transmission-based instructional systems technology model and an emerging atheoretical eclecticism. The excitement of re-grounding the field in constructivism comes through in passages such as the following:

The overarching goal of [the constructivist] approach is to move the learner into thinking in the knowledge domain as an expert user of that domain might think. Hence, [instructional] designers operating under these assumptions must identify the variety of expert users and the tasks they do. For example, our goal should not be to teach students geography principles or geography facts, but to teach students to use the domain of geographic information as a geographer, navigator, cartographer, etc. might do. (Bednar et al., 1991, p. 93)

Despite the plausibility of a link between educational technology and constructivism, however, up to now vision statements have outnumbered practical implementations and far outnumbered empirical validations. In the five years following Bednar et al.'s (1991) much-cited paper, Bednar could not name a specific study applying and testing its proposals (personal communication, 1996). If constructivism is to be taken seriously as a new paradigm for the information age, then vision statements must at some point give way to a program of empirical research leading to a database of the learner types, conditions, and domains in which constructivist approaches have been useful.

In some domains, of course, some degree of constructivism has always been the norm. In physical science subjects, even in secondary school, theory construction and testing is the main agenda in the laboratory. In any domain, the doctoral degree is awarded for transforming low-level data into high-level knowledge. So the notion of increasing the emphasis on constructivist learning refers mainly to expanding constructivism into domains or cultures where transmission models have dominated. I propose that a domain suitable for testing the expandability of constructivist principles will meet the following criteria. It will be a domain where:

• transmission-based instructional models have traditionally dominated but have not produced the desired learning;

 
• an empirical methodology is available to measure quantities and qualities of learning;

 
• there is an in-principle argument that a constructivist approach could produce the desired learning, and yet constructivism does not occur naturally;

 
• a model of expert knowledge construction is available to serve as a basis for instructional design.

VOCABULARY ACQUISITION AS TEST DOMAIN

Among the small number of empirical studies cited by Brown et al. (1989) as support for constructivism was a study of vocabulary acquisition conducted by Miller and Gildea (1987). This study showed that children learned little about words from dictionary definitions (preconstructed and transmitted knowledge), but learned much from wresting the meanings of new words out of natural sentence contexts (knowledge constructed from data), particularly if the goal was to use the words productively in novel sentences (transfer). Admittedly, Miller and Gildea's participants were children learning English, their first language; however, their finding has been replicated with both adults (Gildea, Miller & Wurtenberg, 1990) and adults learning a second language (Nesi & Meara, 1994). Second language vocabulary acquisition meets all the domain criteria proposed above.

Dominance of Transmission Model

There is little doubt that most conscious word learning of second-language learners in and out of classrooms involves the consultation of dictionaries, a classic source of pre-encoded knowledge. Learners' preference for definitions over other word information is well attested in the research (e.g., Bland, Noblitt, Armstrong, & Gray, 1990). And yet learning from definitions alone has proven ineffective, as shown by many studies in addition to Miller and Gildea's (1987), including McKeown (1993). Lexical knowledge acquired from a definition tends to remain inert and untransferable to novel contexts (unless, of course, one is merely using the definition as confirmation).

A possible reason for this is the way definitional information is logically organized. A definition starts by categorizing the look-up word at the next higher order of generality, that is, at the next lower order of frequency, so that words are explained via other words even less likely to be known ("a car is a vehicle which..."). If learners are looking up "car," what hope they will know "vehicle"? In other words, the traditional genus-and-differentia structure of definitions is inherently unsuited to learning the high-frequency words of a language which typically occupy second-language learners.

Empirical Methodology Available

Miller and Gildea's (1987) original framework comparing definitional and contextual treatments on a transfer measure has become a standard in lexical research and has provided much reliable and useful information. The treatment variables have been further specified (types of definitions, numbers and types of contexts), but the outcome variable of abiding interest has been a transfer measure (e.g., Prince, 1996), although of course operationalized in different ways.

In-Principle Case

A wealth of studies support a constructivist approach to word learning. Extensive reviews by Mezynski (1983) and Stahl and Fairbanks (1986) conclude that a word is best learned through meaningful encounters in several natural contexts, with or without the aid of some sort of definition. The resulting knowledge will then be productive and transferable, as measured by its contributing to the comprehension of a novel text in which the word features. There are several mechanisms that could explain the link between multi-contextual learning and transferable lexical knowledge. One is that the extra effort of drawing an inference makes the learning memorable (Hulstijn, 1992). Another is that meeting a word in several contexts paradoxically decontextualizes its mental representation, facilitating the computation of novel instantiations (Sternberg, 1987). Still another is that all cognitive and motor skills are schematized at an abstract and hence generalizable level if they are practiced in varying situations (Schmidt & Bjork, 1992).

No Natural Occurrence

With the problems of definitional learning and the benefits of contextual learning so clear, what prevents language learners from embracing data-driven vocabulary acquisition practices without urging? Of course there is a catch to contextual learning. The information in a given natural context is typically incomplete or even misleading, as shown in a number of studies (beginning with Beck, McKeown & McCaslin, 1983). It is only over a lengthy period of time, such as the span of a normal childhood, that enough words are met in enough contexts to make contextual learning feasible. Even so, how children learn as much about words as they do remains a cause for speculation (Landauer & Dumais, 1997). To detail the pace of natural lexical growth, Nagy, Herman and Anderson (1985) showed that the probability of learning any word from a single contextual encounter was only about .15, but that this was adequate to account for the measured lexical growth of school-aged first-language learners on the basis of an average reading intake of 1 million words per year for 10 years. However, as Meara (1988) has argued, these figures are hardly applicable to second-language learning, where learners are more likely to encounter in the region of a few thousand words per year and the years available for learning are not ten but one or two.

Martin (1984) summarizes the problem produced by the very different conditions of first and second language learning:

The luxury of multiple exposures to words over time in a variety of meaningful contexts is denied to second and foreign language students. They need prodigious amounts of information within an artificially short time How can this enormous amount of information be imparted? ( p. 130)

Learning a second language involves learning thousands of new words. Definitions do not help language learners much, and yet the time for natural contextual learning is demonstrably absent. This problem has been regularly noted by theoreticians of second-language acquisition. Many years ago, the linguist J.B. Carroll (1964) expressed the wish that some form of vocabulary instruction could be found that would mimic the effects of natural, data-driven, contextual learning, except more efficiently. The problem was restated 25 years later in essentially the same form by second-language theorist Krashen (1989):

It ... appears to be the case that vocabulary teaching methods that attempt to do what reading does--give the student a complete knowledge of the word--are not efficient, and those that are efficient result in superficial knowledge. (p. 450)

The similarity in phrasing across these quotations is striking, especially the recurring interest in some "more effective" or "more efficient" approach to vocabulary growth. There is clearly room for a new approach to instruction in this area, some way of making contextual data more accessible to learners.

An Expert Model

Finally, the domain of vocabulary acquisition brings with it an expert model of knowledge construction. The experts in working out the meanings of words from context are, of course, dictionary writers themselves. Like second language learners, lexicographers need to process large amounts of lexical information in unnaturally short time periods, but unlike most language learners they have developed tools and strategies to help them do this.
 
 

A PROFESSIONAL MODEL OF WORD LEARNING

Modern lexicography is a data-driven, empirical science. Lexicographers piece together the meanings of words on the basis of large numbers of examples from language users' speech and writing, that is, on the basis of inference from multiple contexts. This was not always the case. Traditionally, lexicographers established the meaning of a word on the basis of what some authority had said it meant, or what it had meant in Greek or Latin, or on the lexicographer's own instinctive understanding--in other words, on the basis of other definitions. But in the 19th century, in keeping with empirical developments in the physical sciences, lexicographers began to examine texts and speech transcripts instead of other definitions to determine the meanings of words. The famous case is the Oxford English Dictionary, or OED (1884/1928), compiled between 1857 and 1927, which included for each word several authentic example sentences from both old and contemporary sources, displaying clearly the basis for the proposed definition and acknowledging implicitly that meaning was not necessarily static.

Unfortunately, the amount of linguistic data that early modern lexicographers could consult was limited by the prevailing text-processing technology. The telling portrait is Oxford's James Murray, working on the OED project at some point in its history in his garden shed "scriptorium" (Murray, 1977) surrounded by hundreds of cartons of illustrative hand-written quotations submitted over a 70-year period by thousands of amateur wordsmiths throughout the English-speaking world. The empirical approach to dictionary writing has since been made simpler by text-processing technology (see Ooi, 1998, for a review of current procedures). All the information in Murray's scriptorium can now be housed in a child's computer, and the world provides an unending flow of electronic text in all domains and modalities that can be mined for clues as to what speakers and writers think their words mean. Ever larger text corpora have been collected, sampled, and analyzed, starting with Kucera and Francis's (1967) corpus of 1 million words and ending, for the moment, with the British National Corpus (1997) at 100 million words.

It is the vastness of the modern text corpus that puts lexicographer and language learner into similar positions. Like learners who cannot meet words often enough within a year or two to integrate stable meanings, lexicographers do not have the time to read through gigabytes of text searching for all the recurrences of a word or phrase that together constitute its meaning. Instead, they must find ways to aggregate thinly distributed data, and for this they have developed a computer program called a concordance. This program is capable of searching through enormous amounts of text and assembling all the occurrences of particular words, phrases, or structures. In Figure 1, a concordance program (Antworth, 1993) has assembled instances of the word "Alice" from Alice in Wonderland. If any instance seems particularly interesting, the user can access the full original context.

Figure 1: Concordance: computer-aided contextual inference

There is no reason in principle that some version of this technology could not solve the longstanding problem of Carroll (1964) and others, discussed above, by efficiently delivering many years' worth of natural contextual exposures within the typical year or two of second language learning.

However, it could not be taken for granted that professional corpora and concordances could simply be put into the hands of learners unmodified. The tools of corpus analysis are not simple to operate, nor their outputs simple to interpret (understanding the interface developed to make sense of the British National Corpus itself requires a training course). To be useful to language learners in vocabulary acquisition, such tools would need to be extensively modified, focused on specific tasks, and reconsidered from the motivational aspect. Even then, it could not be taken for granted that the multiple contexts gathered by a concordance program would have the same learning power as multiple contexts met within a more natural time frame and goal structure. Whether they can is an empirical question, to be addressed in the following study.
 

METHOD

Participants

The participants in this training study were 58 male and female first-year students at Sultan Qaboos University in Oman in 1993-1996. These students were studying English in preparation for academic courses to be taught in English in business, commerce, and information technology. Their learning was consequential (failure was possible); they had been trained in a strongly transmission-based style of learning (much as described by Gardner, 1993, under the heading "traditional education"); and their approach to learning had been explicitly identified as an impediment to their academic progress.

Since the University opened in 1986, the students' learning habits have been the subject of several studies. In one of these, Barwani, Yahya and Ibrahim (1994) asked faculty members to rate 34 learning-related skills and attitudes on two dimensions: (a) importance to academic success and (b) availability in students. Responses were entered into a factor analysis, and only one of the 34 skills came out as both desirable and available: "ability to memorize and recall information." Highly desirable, but unavailable, was "knowing how to learn." It was an explicit if ill-defined concern of the University to encourage the students toward a more independent approach to their learning.

The students' memory-based approach to learning was particularly evident in their approach to acquiring the vocabulary of English, their new medium of instruction. Their main learning strategy was to use a dictionary to write out long lists of English-Arabic translation equivalents.

Needs analysis. Vocabulary acquisition was arguably the main task confronting these students in their first year. Several studies have shown that of the many components of knowing a second language, vocabulary size correlates most with subsequent academic achievement (e.g., Saville-Troike, 1984). An on-site survey by Arden-Close (1993a, p. 251) of British and American lecturers teaching English-medium academic courses concluded that 3language problems in these lectures are seen as almost exclusively vocabulary problems.2 The specific hurdle facing the first-year students was to pass a standardized test of English known as the Preliminary English Test (PET) (Cambridge, 1990), which is written from a lexical base of the 2,387 highest frequency words of English (as determined by Hindmarsh, 1980). The students' chances of doing well on any test assuming knowledge of 2,387 words was predictable from their performance on Nation's (1990) Vocabulary Levels Test, which students in the College of Commerce wrote on entering the University. The majority of students knew between 500 and 1,000 English words. Assuming a minimum goal of 2,000 words at the end of a year's instruction to be in good shape for their test, even the most advanced among these students was faced with a learning task in the range of 1,000 new words. To see this task in perspective, it is roughly double the learning of comparable European learners (who gain on average 550 new words per year, as calculated by Milton & Meara, 1995).

As the present study began, the PET had been in use at Sultan Qaboos University for two years. The failure rate had been high, with the lowest scores in the reading section of the test. Reading has been shown to be the skill area most dependent on broad vocabulary knowledge (Anderson & Freebody, 1983). But, as already discussed, vocabulary knowledge contributes to reading comprehension mainly when gained through natural contextual encounters, that is, through reading.

Learning words with a dictionary was not a very effective strategy for these students. In an on-site replication of Miller and Gildea's (1987) study, Horst (1995) found that despite their enthusiasm for definitions these students typically misinterpreted definitional information just as the children in the original experiments had done. Yet, unlike Miller and Gildea's children, these students did not have a compensating ability for inferring word meanings from context when definitions were unavailable. This was shown in a talk-aloud study in this same population by Arden-Close (1993b). Protocols of students inferring meanings for new words revealed that most of their inferences were erroneous, being typically based on word-internal information such as one English word sounding like another, or even like an Arabic word. Broader linguistic or situational contexts were rarely considered. In other words, neither of the two obvious routes to building a second lexicon--definitions or contexts--held much promise.

What could an expert model offer? There is a large gap between knowing that students would learn more if they constructed their lexical knowledge from textual data, and providing them with the means for doing so. Instructors and reading-skills textbooks have long exhorted learners to adopt data-driven approaches to vocabulary acquisition, but the majority have always declined. In fact, language learners may have good reasons for balking at the suggestion they forswear their dictionaries.

Informal discussions with successive cohorts of students at Sultan Qaboos University suggest they know instinctively some interesting facts about vocabulary acquisition. They suspect that the natural contextual learning processes of first-language acquisition operate too slowly to help them much (as confirmed by Nagy et al., 1985). They suspect that natural contexts often contain inadequate and misleading information about what words mean (as confirmed by Beck et al., 1983). They realize that given the amount of reading they are likely to do in English, they will not meet new words enough times for significant learning to take place. Saragi, Nation and Meister (1978) found almost no learning from fewer than five encounters. The students sense that despite their instructors' exhortations about using context, their instructional materials have not really been designed to help them in this. Wodinsky and Nation (1988) bear this out, finding that even in vocabulary-controlled readers, conditions are far from optimal for contextual learning. Finally, they have gathered from several rounds of classroom guessing-from-context activities that their present vocabulary base is inadequate for inferential learning. Laufer (1989) finds little reliable learning from texts with more than 1 word unknown in 20. For learners knowing only 1,000 words, a typical text will present as many as 5 unknown words in 20.

However, the lexicographer's concordance tool could in principle overcome some of the learners' reservations about learning from context. With its data-aggregating ability, a concordance can reduce the amount of time required for several contextualizations of a word to present themselves. It might also reduce the power of unclear contexts to mislead by giving several to choose from. With enough sentence contexts available for any word, chances are good that in at least one of them only 1 word in 20 will be unknown, even for a learner with only 1,000 words.

On the negative side, a concordance is difficult to read; its texts are reduced in size and coherence; there is little opportunity for the normal flow of natural reading to build up (see Figure 1); and the on-screen text-to-space ratio breaks normal standards of text design (Waller, 1991). To the instructional designer, it is far from obvious that a corpus-based tutor will survive the transition from the lexicography unit to the language unit.

Apparatus

A learner corpus and concordance

Several projects involving the educational uses of concordances had already been undertaken at Sultan Qaboos University (Cobb, 1997a; Flowerdew, 1993; Stevens, 1991), yet not to the point of putting concordances into the hands of learners themselves. Several corpora and concordance interfaces were pilot tested with a number of intact classes of first-year students in 1993-95. It quickly became clear that neither a standard corpus nor a standard interface would be interesting for these learners to use. Even corpora specifically designed for language learning, such as Microconcord (Oxford english Software, 1993) or Cobuild's Bank of English (Collins-Cobuild, 1997), were far more linguistically complex than these students were able cope with given their English proficiency level. The in-house corpus eventually assembled for this study was a 50,000-word collection of the students' own language course materials, chosen on the principle that most of the 2,387 words they needed to know were actually present in their materials in sufficient frequency, just too widely distributed for effective learning within the time available. (Corpus building procedures are discussed in Cobb, 1996.)

As for the interface, observation and feedback suggested that keyboard entry posed an obstacle for the students, most of whom were inexperienced typists and poor English spellers, and that the screen layout of a standard concordance output did not make different kinds of textual information visually salient. As for instructional design, the standard interface did not make it clear exactly what learners were supposed to do in a concordancing session--which words to investigate, or what to do with lexical information once they had assembled it.

It became clear that a concordance program for the early stages of language learning should do the following:
 

identify which words to investigate (in this case items on the high-frequency PET word list);

provide a means for learners to take away lexical information in hard copy (or else most of the lab hour was spent transcribing from the screen);

 

build in motivation for considering each word in several sentence contexts (since not all learners saw the point of this).

... [which] mesh in with software that helps lexicographers in the actual construction of the dictionary, allowing them, for instance, to cut and paste examples straight from the corpus into the appropriate section of an entry, screen definitions for compatibility with a controlling restricted vocabulary... In its most sophisticated form, the lexicographer's workstation is an integrated computer system in which the lexicographer can switch between all the components--the corpus, concordancing, retrieval and statistical software, and the software which guides the gradual compilation of the dictionary--at the click of a mouse button. (p. 106)

Since none of the concordance packages commercially available met these criteria, I set about designing the learner version of Meijs's workstation myself, drawing on local and international expertise and support. The main concordance engine was a shareware Pascal routine called TEXAS (Zimmerman, 1988), and the platform for interface development was Apple's Hypercard.

Design of PET-2000, a dictionary toolkit. Figure 2 shows the concordance interface resulting from several cycles of formative evaluation, an interface simplified for language learners yet preserving key features of the lexicographer's toolkit. The corpus and concordance are fully integrated with an easy-access list of the 2,387 PET words. Clicking on the letter A at the top of the screen brings all the words beginning with the letter a into a scrolling window; clicking on any word brings all instances of that word into a second window as contextualized concordance lines; clicking on any concordance line expands that line into its source text in a third window, showing the target word in upper case and highlighting the immediate sentence. Used words are underlined so that learners can keep track of what they have done. The three windows are in three colors and three text fonts so that different information types are visually distinct. A large number of words appear on the screen, but a maximum of empty space has been preserved, and 12- or 14-point fonts have been used wherever possible. The program works fast enough on the Macintosh LCIII machines in the students' lab for them to see clearly the results of their actions.
 
 

Figure 2 PET-2000 interface

PET-2000 thus allows a great many words to be viewed in several contexts in a short time, providing a zoom-lens on a large amount of distributed data. Following Meij (1996), I linked the concordance interface to a database that students kept on their floppy disks, where they could store their findings simply by clicking Enter in Word Stack. An entry in the database was created for the target word along with whatever portion of the source text the student had selected, either the default sentence or more (or less) contextual information as desired. Additional context sentences could be selected for the same word or for different forms of the word. Figure 3 shows a database entry assembled from the concordance information shown in Figure 2. There is a space for typing a definition in either English or Arabic.

Figure 3 Personal Word Stack

Procedure

Conditions of use of PET-2000.The students' second-term reading course was expanded to include a PET-words vocabulary module. The entire 2,387-word list was installed in the concordance interface, shown in the window at the top of Figure 2, and the students were assigned roughly 200 words from the list to be learned every week (e.g., all the words starting with C). The list was available only on the computers, and one computer lab-hour was set aside per class per week for this work. Twenty randomly selected to-be-learned words were tested weekly in the classroom. The students were advised to look through the roughly 200 new words every week for 12 weeks, decide which ones they knew inadequately, and send these to the database with one or more examples from the corpus; add definitions if desired, and print up the session's work as an installment in a growing personal glossary (shown in Figure 4).

Figure 4 Constructing a dictionary

Learners worked individually or in groups. Most worked individually, although later they often photocopied pages from other students' glossaries to supplement their own. No attempt was made to prevent them from consulting dictionaries to fill in the definition slot in the database if they wished, but the design ensured they had met the word in several contexts before that. The student's work in Figure 4 shows the examples and Arabic definitions she has added for one page in her glossary. It may seem grandiloquent to refer to these definitions as knowledge constructions, since in most cases they are about the same as could be found in any small dictionary. However, the majority of participants reported that when they had found even one totally clear example for a word, then its meaning seemed obvious and they did not need a dictionary.

The students also reported that they normally checked several contexts before selecting one for the database, and the system's user tracking confirmed this (for more details on system use, see Cobb, 1996). Fewer than half the examples selected were simply the first one listed in the concordance. Several aspects of the student's entries in Figure 4 hint at rather sophisticated word-learning strategies: Word-family information has been noticed and included ("employee" as well as "employ"). Two senses of "engaged" have been recorded, and several possible Arabic translations have been entered for some of the words, suggesting a learning strategy more interesting than the usual one-to-one translations produced by "the naive lexical hypothesis" (Bland et al, 1990, p. 440). Most of the context sentences the student has chosen are fairly clear illustrations of an important sense of the target word.

Testing the tutor. The learning effects of PET-2000 were assessed during the second run of the final version of the system, at a time when the students had been learning English for about five months. Of the several groups using PETï2000, four intact groups at two proficiency levels were randomly selected for testing. The first level consisted of two groups (n=17 each) of lower intermediate students (meaning they knew on average 1200 English words, as established by Nation's, 1990, Vocabulary Levels Test). The second consisted of two groups (n=12 each) of upper intermediate students (with an average of 1500 words).

At each level, groups were randomly assigned to experimental and control conditions. Experimental group participants used the concordance software as described above; control group participants used a modified version of the software to send items with no examples to their databases for annotation with Arabic translations from their (off-line) dictionaries. In each condition, there were a low-intermediate and high-intermediate group, so that independent comparisons could be made at two levels. Within each level, the classes selected for the study were equal in terms of both definitional vocabulary size and ability to transfer vocabulary knowledge to a novel context (see Table 1). Participants had been assigned to groups randomly by the institution (except that proportion of males to females was balanced in each class). Context users and definition copiers spent roughly the same amount of time on their PET-2000 work, 45 minutes per student per week for 12 weeks, with very little variance, according to both informal observation and the program's on-line time record. There was of course no control on the amount of time participants studied their glossaries or did further work on them after class.

Measures. All participants were tested before and after their 12 weeks of vocabulary work using the same two-part vocabulary test. The first part consisted of the 2,000-level of Nation's (1990) Vocabulary Levels Test, which asked students to select brief definitions for 18 words randomly sampled from the 2,000 most frequent English words. This widely used test is arguably the most reliable vocabulary size measure currently available (Schmitt, 1995). The second part of the test was a gapped passage, which asked examinees to fit 15 supplied words from the same frequency level to gaps in two novel texts of about 250 words each (see Figure 5).

Figure 5 Pretest-posttest (partial) (Part A from Nation, 1990)
 

 The two sections of the test were intended to assess two kinds of lexical knowledge: (a) relatively superficial definitional knowledge, and (b) the more complex knowledge required to recognize the appropriateness of new words in novel extended contexts. Gapped passages (also known as rational cloze or systematic deletion passages), particularly when longer than a sentence or two, have been endorsed by reading theorists as a rich measure of lexical acquisition (e.g., Beck & McKeown, 1991), and such passages have the additional advantage of being a task format well known to language learners. This measure is a suitable test of "flexible, transferable, and useful" word knowledge (to quote the opening paragraph of the abstract): it tests flexibility, since in almost every extended novel context a word's meaning is instantiated in a slightly non-canonical way (Anderson & Ortony, 1975); it clearly tests transfer; and it tests usefulness, since the intended use of lexical knowledge is to comprehend novel full-length texts. It is almost a given in lexical research that learners will be more successful with definitional than with transfer tasks. The test was given two weeks before the beginning of the training period and then again two weeks after the end. No feedback was given after either testing session, and there was no indication that participants remembered the test in any detail when they encountered it a second time.

Research design. The study was organized around a repeated measures factorial design, the factors being 2 (Treatments) x 2 (Skill Dimensions) x 2 (Levels) at two points in time (pre- and post-training). It was expected that the two treatments (contexts vs. definitions) would produce roughly equal gains on the definitional knowledge dimension, but that students who had worked with contexts would be significantly stronger on the transfer dimension. Pretest and posttest scores for each skill dimension were entered into a separate repeated measures analysis of variance (ANOVA), with pretest and posttest scores as dependent variables and level and treatment as independent variables. If the experimental prediction was borne out, there should be a significant time-treatment interaction on the transfer task, but not on the definitions task.
 
 

RESULTS

On the definitional measure, there was a main effect for time, F(1,54)=6.74, p<.05, showing overall pretest scores (M=69.53) were lower than posttest (M=75.91). However, this effect was unrelated to treatment and probably reflected normal acquisition rates. A gain of just over 6 percent on the 2000-level of the Vocabulary Levels Test represents a gain of 120 words, which in a period of 12 weeks means the learners were learning and retaining about 10 new words every week (in keeping with Milton and Meara's, 1995, finding of 550 words per year). Of course, this figure tells us little about what the learners were able to do with this knowledge.

On the transfer measure, there was a similar although larger main effect for time, F(1,54)=19.48, p<.001, showing overall pretest scores (M=64.84) were lower than posttest (M=74.03). But on this measure there was also a significant interaction between time and treatment, F(1,54)=6.24, p<.05, suggesting differential contributions to the gain by condition (see Table 1). The control students' pretest-posttest gain was 3.83%, compared to the learner-lexicographers' 14.55%. The significance of this difference was confirmed in a post hoc Tukey honestly significant difference (HSD) test of multiple comparisons, the results of which were as follows: control and experimental group means were  equal at pretest; control group means were equal from pretest to posttest; control and experimental group means were different at posttest; and experimental group means were different from pretest to posttest, a (1,56)=10.69, p<.001. There was no interaction between treatment and level.
 

Table 1 Pre-post results by level and condition
 

		Definitions Task						Novel Text Task
	Control			Experimental			Control			Experimental
	Pre	Post	Gain	Pre	Post	Gain	Pre	Post	Gain	Pre	Post	Gain
			Lower Intermediate
M	65.24	71.94	6.7*	65.53	74.4	8.87*	60.24	62.76	2.52	60.65	74.12	13.47**
SD	15.38	13.41		12.40	13.81		19.33	17.08		17.80	15.00
			Upper Intermediate
M	75.17	79.58	4.41*	75.67	79.92	4.25*	71.42	77.08	5.66	70.75	86.83	16.08**
SD	11.18	12.23		10.80	12.00		12.14	10.66		12.35	8.90
*p<.05  **p<.001

DISCUSSION

It seems clear that the lexicography group learners were thinking about new words in a way that made their knowledge more transferable to novel contexts. In itself, this finding is not entirely surprising, since it replicates a standard off-line finding. It has long been known that interpreting new words in multiple contexts is the main precondition for producing transferable word knowledge (Mezynski, 1983; Stahl & Fairbanks, 1986). However, as stated previously, it could not be taken for granted that the multiple contexts gathered by a concordance program would have the same learning power as the multiple contexts met within a more natural time frame and goal structure. The present study shows that the contexts need not be particularly lengthy, continuous, or encountered over extended periods of time in order to produce a significant measure of transfer, and hence can be assembled and delivered by a computer program without losing their learning power. In other words, concordancing can considerably abbreviate the natural learning process where there is a reason to do so, and hence may answer the wish of Carroll (1964) and others for vocabulary instruction that mimics the effects of natural contextual learning except more efficiently.

In a recent discussion of cognitive efficiency and media (Cobb, 1997b), I argued that it could be useful to compare different ways of presenting the same information with respect to speed or ease of learning, or efficiency, and here I propose the concordance as an instance of an efficient medium. A medium is efficient to the extent that it performs some of the work of learning. The main work performed by the concordance is to reduce the memory burden of natural contextual learning, which is normally beset by cycles of partial learning and forgetting. There is no suggestion that meeting contexts in a concordance listing produces superior knowledge to meeting them spaced out on printed pages, just that it produces some of the same type of knowledge but in a reduced time frame.

In the PET-2000 project, it seems clear that constructivism provided the general perspective and lexicography the specific tools for an effective instructional design. In vocabulary acquisition, several constraints that are possibly domain specific come together to make a learner-as-scientist approach useful: the nature of the data (voluminous and widely distributed), the goals of learning (to transfer knowledge to novel tasks), the conditions of learning (a foreshortened time frame), and the limitations of transmission-based alternatives (the way definitional information is organized). Whether expert models and tools will prove similarly useful in other domains can not be taken for granted. The present study does suggest one generalization, that in domains where experts and learners share an important need, in this case for data aggregation, then some version of the expert's tools and procedures may be useful to learners.
 
 

REFERENCES

Anderson, J.R., Reder, L.M. & Simon, H.A. (1995). Applications and misapplications of cognitive psychology to mathematics education. [On-line.] Available: http://www.psy.cmu.edu/~mm4b/misapplied.html.

Anderson, J.R., Reder, L.M., & Simon, H.A. (1996). Situated learning and cognition. Educational Researcher, 25 (4), 5-11.

Anderson, R.C. & Freebody, P. (1983). Reading comprehension and the assessment and acquisition of word knowledge. In J.T. Guthrie (Ed.), Comprehension & teaching: Research reviews. Newark, DE: International Reading Association.

Anderson, R.C. & Ortony, A. (1975). On putting apples into bottles--a problem of polysemy. Cognitive Psychology, 7, 167-180.

Antworth, E. (1993). Conc 1.76: Concordance generator program. [Computer program.] Academic Computing Department, Summer Institute of Linguistics, Dallas TX.

Arden-Close, C. (1993a). Language problems in science lectures to non-native speakers. English for Specific Purposes, 12, 251-261.

Arden-Close, C. (1993b). Learners' strategies for inferring the meanings of unknown words. Unpublished manuscript. Sultan Qaboos University, Muscat, Oman.

Ausubel, D. P. (1968). Educational psychology: A cognitive view. New York: Holt, Rinehart and Winston.

Barwani, T., Yahya, A., & Ibrahim, A. (1994). Basic skills and attitudes for university study as perceived by faculty members at Sultan Qaboos University. Internal Report, College of Education and Islamic Sciences, Sultan Qaboos University.

Beck, I.L., McKeown, M.G., & McCaslin, E.S. (1983). Vocabulary development: All contexts are not created equal. The Elementary School Journal, 83 (3), 177-181.

Beck, I.L. & McKeown, M.G. (1991). Conditions of vocabulary acquisition. In R. Barr, M.L. Kamil, P. Rosenthal, & P.D. Pearson (Eds.), Handbook of Reading Research, Vol II (pp. 789-814). New York: Longman.

Bednar, A.K., Cunningham, D., Duffy, T.M., & Perry, J.D. (1991). Theory into practice: How do we link? In G.L. Anglin (Ed.), Instructional technology: Past, present, and future (pp. 88-101). Englewood, CO: Libraries Unlimited.

Bereiter, C. (1994). Constructivism, socioculturalism, and Popper's World 3. Educational Researcher, 23 (7), 21-23.

Bland, S.K., Noblitt, J.S., Armstrong, S., & Gray, G. (1990). The naive lexical hypothesis: Evidence from computer-assisted language learning. The Modern Language Journal, 74, 440-450.

British National Corpus (1997). [On-line.] Available: <http://info.ox.ac.uk/bnc>.

Brown, A. L., & Campione, J. C. (1994). Guided discovery in a community of learners. In K. McGilly (Ed.) Classroom lessons: Integrating cognitive theory and classroom practice (pp. 229-270). Cambridge: MIT Press.

Brown, A.L., Bransford, J.D., Ferrara, R.A., & Campione, J.C. (1983). Learning, remembering, and understanding. In J.H. Flavell and E.M. Markman (Eds.), Mussen's handbook of child psychology: Vol. 3 Cognitive development (pp. 77-166). New York: Wiley.

Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18 (1), 32-42.
Cambridge University. (1990). Preliminary English Test. Local Examinations Syndicate: International examinations.

Carroll, J.B. (1964). Words, meanings, & concepts. Harvard Educational Review, 34, 178-202.

Chi, M.T.H., Glaser, R., & Farr, M.J. (1988). The nature of expertise. Hillsdale, NJ: Erlbaum.

Cobb, T. (1996). From concord to lexicon: Development and test of a corpus-based lexical tutor. Unpublished doctoral dissertation. Concordia University, Montreal.

Cobb, T. (1997a). Is there any measurable learning from hands-on concordancing? System 25, 301-315.

Cobb, T.M. (1997b). Cognitive efficiency: Toward a revised theory of media. Educational Technology Research & Development, 45 (4), 21-35.

Collins-Cobuild (1997). Bank of English. [On-line.] Available http://titania.cobuild.co.uk.

Dewey, J. (1938.) Logic: The theory of inquiry. New York: Holt.

Duffy, T., & Jonassen, D.H. (1991). Constructivism: New implications for instructional technology? Educational Technology, 31 (5), 7-12.

Duffy, T., & Jonassen, D.H. (1992). Constructivism and the technology of instruction: A conversation. Hillsdale, NJ: Erlbaum.

Flowerdew, J. (1993). Concordancing as a tool in course design. System 21, 231-244.

Gardner, H. (1993). Frames of mind: The theory of multiple intelligences (2nd Ed.). New York: Basic Books.

Gildea, P.M., Miller, G., & Wurtenberg, C. (1990). Contextual enrichment by videodisc. In D. Nix & R. Spiro (Eds)., Cognition, education, multimedia: Exploring ideas in high technology. Hillsdale, NJ: Erlbaum.

Hindmarsh, R. (1980). Cambridge English Lexicon. Cambridge: Cambridge University Press.

Horst, M.E. (1995, March). Who get summit mark in the class? Productive use of definitions and examples in beginning learners of English. Paper presented at TESOL Arabia, Al-Ayn, United Arab Emirates.

Hulstijn, J.H. (1992). Retention of inferred and given word meanings: Experiments in incidental learning. In P.J. Arnaud & H. Béjoint (Eds.), Vocabulary and applied linguistics (pp. 113-125). London: Macmillan.

Krashen, S.D. (1989). We acquire vocabulary and spelling by reading: Additional evidence for the input hypothesis. The Modern Language Journal 73, 440-464.

Kucera, H., & Francis, W.N. (1967). A computational analysis of present-day American English. Providence RI: Brown University Press.

Lajoie, S.P., & Lesgold, A. (1989). Apprenticeship training in the workplace: Computer-coached practice environment as a new form of apprenticeship. Machine-Mediated Learning 3, 7-28.

Landauer, T.K., & Dumais, S.T. (1997). A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, & representation of knowledge. Psychological Review 104, 211-240.

Laufer, B. (1989). What percentage of text-lexis is essential for comprehension? In C. Lauren & M. Nordman (Eds.), Special language: From humans thinking to thinking machines (pp. 316-323). Clevedon, UK: Multilingual Matters.

Martin, M. (1984). Advanced vocabulary teaching: The problem of synonyms. Modern Language Journal 69, 130-137.

McCloskey, M. (1983). Naive theories of motion. In D. Gentner & A.L. Stevens (Eds.), Mental models. Hillsdale, NJ: Erlbaum.

McKeown, M. (1993). Creating effective definitions for young word learners. Reading Research Quarterly 28, 17-31.

Meara, P. (1988). Learning words in an L1 and an L2. Polyglot, 9 (3), D1-E14.

Meijs, W. (1996). Linguistic corpora and lexicography. Annual Review of Applied Linguistics 16, 99-114.

Merrill, M.D. (1991). Constructivism and instructional design. Educational Technology, 31 (5), 45-53.

Mezynski, K. (1983). Issues concerning the acquisition of knowledge: Effects of vocabulary training on reading comprehension. Review of Educational Research 53, 253-279.

Miller, G.A., & Gildea, P.M. (1987). How children learn words. Scientific American, 257 (3), 94-99.

Milton, J., & Meara, P. (1995). How periods abroad affect vocabulary growth in a foreign language. ITL Review of Applied Linguistics 107/108, 17-34.

Murray, K.M.E. (1977). Caught in the web of words: James Murray and the Oxford English Dictionary. New York: Oxford University Press.

Nagy, W.E., Herman, P.A., & Anderson, R.C. (1985). Learning words from context. Reading Research Quarterly 20, 233-253.

Nation, P. (1990). Teaching and learning vocabulary. New York: Newbury House.

Nesi, H. & Meara, P. (1994). Patterns of misinterpretation in the productive use of EFL dictionary definitions. System 22, 1-15.

Ooi, V.B.Y. (1998). Computer corpus lexicography. Edinburgh: Edinburgh University Press.

Oxford English Dictionary. (1884/1928). Oxford: Oxford University Press.

Oxford English Software. (1993). MicroConcord corpus collections. Oxford: Oxford University Press.

Phillips, D.C. (1995). The good, the bad, and the ugly: The many faces of constructivism. Educational Researcher, 24 (7), 5-12.

Prince, P. (1996). Second language vocabulary learning: The role of context versus translation as a function of proficiency. The Modern Language Journal 80, 478-493.

Quartz, S.R., & Sejnowski, T.J. (1997). The neural basis of cognitive development: A constructivist manifesto. Behavioral & Brain Sciences 20, 537-596.

Resnick, L.B. (1987). Learning in school and out. Educational Researcher, 16 (9), 13-20.

Saragi, T., Nation, I.S.P., & Meister, G.F. (1978). Vocabulary learning and reading. System 6, 72-78.

Saville-Troike, M. (1984). What really matters in second language learning for academic achievement? TESOL Quarterly 18, 199-219.

Schmidt, R.A., & Bjork, R.A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science 3, 207-217.

Schmitt, N. (1995). An examination of the behavior of four vocabulary tests. Paper presented at Dyffryn Conference on Lexis, Centre for Applied Linguistics, University of Wales at Swansea, April.

Spiro, R.J., Coulson, R.L., Feltovich, P.J., & Anderson, D.K. (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. Paper presented at Tenth Annual Conference of the Cognitive Science Society, Montreal.

Spiro, R.J., Feltovich, P.J., Jacobson, M.J., & Coulson, R.L. (1991). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. Educational Technology, 31 (5), 24-33.

Stahl, S.A., & Fairbanks, M.M. (1986). The effects of vocabulary instruction: A model-based meta-analysis. Review of Educational Research 56, 72-110.

Sternberg, R.J. (1987). Most vocabulary is learned from context. In M.G. McKeown & M.E. Curtis (Eds.), The nature of vocabulary acquisition (pp. 89-105). Hillsdale, NJ: Erlbaum.

Stevens, V. (1991). Classroom concordancing: Vocabulary materials derived from relevant, authentic text. English for Specific Purposes 10, 35-46.

Waller, R. (1991). Typography and discourse. In R. Barr, M.L. Kamil, P. Rosenthal, & P.D. Pearson (Eds.), Handbook of reading research, Vol. II (pp. 341-380). New York: Longmans.

Wodinsky, M., & Nation, P. (1988). Learning from graded readers. Reading in a Foreign Language 5, 155-161.

Zimmerman, M. (1988). Texas indexer/browser, v. 0.27. [Computer program]. Silver Spring, MD.