International Assessments

Prior to 1980 few teachers utilized information technology (IT) in the classroom. But the global diffusion of personal computers in the 1980s generated considerable interest in educational circles around the world, leading the International Association for the Evaluation of Educational Achievement (IEA) to initiate the first international comparative study of IT or computers in education. This study was named the Computers in Education Study and sometimes called Comp Ed.

IEA Computers in Education Study

Twenty-two countries participated in the first stage of the Computers in Education Study and in 1989 conducted school surveys, as documented by Pelgrum and Plomp. Surveys were conducted in elementary, lower secondary, and upper secondary schools, and within each school sample, questionnaires were completed by the principal, computer coordinator, and several teachers. In 1992 the second stage of the study repeated the surveys of the first stage and added a student assessment, according to Willem Pelgrum and colleagues and Robert E. Anderson.

An assessment was designed to measure the ability of students to generally understand and use information technology. The performance of students in this assessment depended largely on the extent to which school curricula in each country provided opportunity to learn such skills. A number of countries already had instituted an informatics curriculum at the middle-or upper-secondary levels. Perhaps the most important finding of the study was that teachers in general lacked opportunities

TABLE 1

for the type of training that would enable them to integrate technology into their instruction.

The terminology for information technology has changed since the 1980s. Whereas information technology was called computers or IT during that decade, by the late 1990s educators in most countries referred to it as ICT to stand for the phrase information and communication technology. However, in some countries, most notably the United States, educators refer to information technology simply by the word technology.

Second IEA Study

The rapid diffusion of the Internet and multimedia technology during the mid-1990s generated an interest in a new study that among other things could investigate the changes in the curricula and classrooms since IEA's earlier study. The Second International Technology in Education Study (SITES) was initiated in 1996 by the IEA and school surveys were conducted in 1998. The SITES study consists of three modules as summarized in Table 1.

Although the study was approved by the IEA in 1996, the survey data of module 1 were collected in 1998. The module 2 case study visits to the school sites were conducted during 2000 and 2001, and the reports will be released in 2002 and 2003. Module 3 was launched in 2001, but the data for the surveys and student assessments will be collected during 2004, with the results released in 2005 and 2006. Each of the three modules will be described briefly in turn.

School Survey Module. In 1998 data were collected using a questionnaire survey of principals and one of technology coordinators or their equivalents. Twenty-six countries participated by conducting these surveys in one or more of these three school levels: primary, lower secondary, and upper secondary. As reported by Pelgrum and Anderson, this module produced findings on the following phenomena:

• the extent to which ICT is used (and by whom) in education systems across the globe
• the extent to which education systems have adopted, implemented, and realized the results from objectives that are considered important for education in a knowledge society
• teaching practices that principals consider to be innovative, important, effective, and satisfying
• existing differences in ICT-related practices both within and between education systems and what lessons can be learned from this.

The findings on school Internet access were representative of the heterogeneous pattern of cross-national adoption of new ICT practices. Figure 1 shows that while 100 percent of the schools in Singapore and Iceland had access, some countries had only about a fourth of their schools connected. Most of the other countries had connected more than 50 percent of their schools. What is so remarkable about this pattern is that even in populations that do not speak English–the dominant language of the Internet–most of their countries' schools had been connected and many of the students were using the Internet in school. This rapid connection of schools to the Internet occurred within only about five years or less.

FIGURE 1

Case Studies Module. Nearly thirty countries conducted in-depth case studies during the last half of 2000 and the first half of 2001. The focus of this qualitative research is innovative pedagogical practices that use technology (IPPUT). The main purposes are to understand what sustains these practices and what outcomes they produce. To accomplish this investigation, each case study describes and analyzes classroom-based processes and their contexts. These case studies are intended to provide policy analysts and teachers with examples of "model" classroom practices and offer policymakers findings regarding the contextual factors that are critical to successful implementation and sustainability of these exemplary teaching practices using ICT.

The twenty-eight countries participating in this module of SITES were Australia, Canada, Chile, China Hong Kong, Chinese Taipei, Czech Republic, Denmark, England, Finland, France, Germany, Israel, Italy, Japan, Korea, Latvia, Lithuania, the Netherlands, Norway, Philippines, Portugal, Russian Federation, Singapore, Slovakia, South Africa, Spain (Catalonia), Thailand, and the United States. As each country will conduct four to twelve case studies, the total number of cases for analysis is expected to be more than 150.

One noteworthy preliminary finding was that the students used the Internet as part of nearly every innovative practice selected. Another preliminary finding of perhaps greater importance is that the students involved in these innovative pedagogical practices often engaged in activities that could be considered "knowledge management" in that they frequently constructed knowledge products. Typically such activities were called projects and included the tasks of searching, organizing, and evaluating knowledge. For instance, Germany's first case study found that students "turned into providers of knowledge." Portugal's pilot case reported that the teachers wanted their students to be "constructors rather than receptors of mathematical knowledge." In Norway and the USA the case studies found students working collaboratively with ICT tools to complete projects yielding diverse types of knowledge.

Assessment Module. This module builds upon these findings from the leading-edge classrooms of the case studies. Specifically, the school survey, teacher survey, and student assessment will include indicators to determine the difference between the innovative and the typical learning contexts. The study will measure the ICT-supported knowledge management competencies of students, including their abilities to retrieve, organize, critically evaluate, communicate, and produce knowledge. In addition, the study will determine the readiness of schools and teachers to provide a learning environment where students can develop these abilities. In addition, this module will follow up the school survey module by having a school survey administered to principals and ICT-coordinators to measure trends of technology availability and use in schools.

All countries participating in the assessment module will study fourteen-year old students; the target population will be the grade with the most students of age fourteen. An optional population will be the grade with the most ten-year-old students. Each country will be expected to attain a sample of a minimum of 200 randomly selected schools per population. In at least 100 of these participating schools, one intact class will be sampled from all classes in the target grade. In addition to surveying the teacher of the sampled class, three additional teachers will be sampled and surveyed from those teaching the target grade.

Guiding the development of the student assessment is a framework that considers different types of knowledge management and types of tools. The categories of knowledge and tools are shown in Table 2 with the cells that illustrate sample performance tasks.

In the student assessment there will be a paper-and-pencil assessment administered to all students in the sample, and an optional Internet-based performance assessment given to four students in each class, provided that they are Internet-"literate." Students in the sampled class will be administered a survey questionnaire and a short paper-and-pencil assessment during a single class period. The assessment will include a short Internet screening test. Students will not be eligible for participation in the performance assessment unless they pass the screening test. If at least half of the students pass the Internet screening test, then the school will be eligible for participation in the performance assessment.

Despite highly diverse national educational systems around the world, almost every country has established policies regarding ICT in education. SITES in its first two modules found many different approaches across countries to the ICT challenge in education. Yet there are common threads such as widespread and rapidly growing access to the Internet. There is every reason to believe that this trend, as well as the large digital divide across countries, will continue in the early twenty-first century. It is anticipated that the assessment module with its focus on knowledge management will capture significant trends in information technology and the changing role of knowledge in society.