International Assessments

The International Association for the Evaluation of Educational Achievement (IEA), founded in the late 1950s, conducts international comparative studies in which educational achievement is assessed in relation to student background, teacher, classroom, and school variables. At the time of the IEA's founding, there was a growing awareness among international agencies of the role of formal education in social and economical development while indicators for educational "productivity" were lacking. A group of researchers, among them Torsten Husén and his colleagues, decided in the early 1960s to undertake a first study of mathematics achievement in twelve countries to explore the feasibility of international comparative achievement studies. This first study marked the birth of the IEA.

After starting as a group of researchers, the IEA soon became a cooperative of research institutes with a primarily academic research focus. Since the early 1980s the IEA has begun to focus more specifically on the interests of policymakers, and an increasing number of member countries are represented by their ministry of education and no longer by a leading research institute. The current membership of the IEA comprises almost sixty countries from all regions of the world, and these are represented in a policymaking body, the IEA General Assembly, that is supported by the Secretariat in Amsterdam. Each IEA study is managed by three bodies: the International Coordinating Centre (ICC), which is responsible for the conduct of the research at the international level; the International Steering Committee (ISC), which monitors the quality of the research and is responsible for the general policy directions; and the International Study Committee, which consists of the National Research Coordinators (who are responsible for the study at the national level), the ISC, and the ICC.

Over the years the IEA has conducted many survey studies of basic school subjects. Most of the studies were curriculum driven and measure educational outcomes (the attained curriculum) on the basis of an analysis of the "official" curricula (the intended curricula) of the participating countries. All these studies evaluate school and classroom process variables (the implemented curriculum), as well as teacher and student background variables. Examples are the studies of mathematics and science, reading literacy, civics education, and English and French as foreign languages. The IEA also conducts studies that are not curriculum based, such as the Pre-Primary Project and the Computers in Education study. In a typical IEA study, data are collected in the third and/or fourth grade, the seventh and/or eighth grade, and the final year of secondary schooling, although some studies do not include all three populations. IEA's best-known study is the Third International Mathematics and Science Study (TIMSS), conducted between 1992 and 1999 for all three populations, in which more than forty countries participated. This study was designed to assess achievement in mathematics and science in the context of national curricula, instructional practices, and the social (and learning) environment of students.

To allow countries a longitudinal international comparative perspective, the IEA in the late 1990s initiated a basic cycle of studies in which the association studies, in alternating years, mathematics and sciences (through TIMSS–now called Trends in Mathematics and Science Study) and reading literacy (through the Progress in International Reading Literacy Study). Additionally, the IEA conducts other studies such as the Civic Education Study, which was completed in 2002, and the Second Information Technology in Education Study, which started in the fall of 1997.

Purposes and Functions of IEA Studies

IEA's mission is to enhance the quality of education. Its studies have two main purposes: (1) to provide policymakers and educational practitioners with information about the quality of their education in relation to relevant reference countries; and (2) to assist in understanding the reasons for observed differences among educational systems.

Given these purposes, the IEA strives for two kinds of comparisons in its studies. The first one consists of straight international comparisons of effects of education in terms of scores (or subscores) on international tests. The second relates to how well a country's official curriculum is implemented in the schools and achieved by students.

As a result, IEA studies have a variety of functions for educational policymakers, practitioners, and researchers:

• describing the national results in an international context
• analyzing the information about the status of the achievement of pupils against the results of one or more other countries or against the results in the country of interest in an earlier study ("benchmarking")
• analyzing data to contribute to recommendations for changes when and where needed ("monitoring")
• analyzing data with the purpose of understanding the reasons for observed performances either in a national context or within an international comparative perspective
• promoting a general "enlightenment"–that is, there is not a direct link to decisions but rather a gradual diffusion of ideas into the sphere of organizational decision-making

Many national and international reports on IEA studies illustrate the usefulness of IEA studies for educational policy and practice. For example, in Australia, Hungary, Ireland, Japan, New Zealand, and the United States, specific curriculum changes have been attributed to IEA findings.

Considerations in Planning IEA Studies

IEA studies are very complex endeavors and are conducted with attention to quality at every step of the way. The first question to be answered in planning an IEA study is "what questions do we want to address through this study." The leading research questions become very compelling because there are a variety of competing perspectives in realizing a study. In order to obtain valid and useful data and indicators, high-level scientific and technical standards have to be met for each component of the study, such as the development of a conceptual framework; determining target populations; curriculum analysis; instrument development (including pilot testing, translation, etc.); sampling; data collection, cleaning and file building; quality control in participating countries of each component; data analysis; and report writing.

Many countries participate in IEA studies (e.g., in TIMSS more than forty countries), and according to its mission the IEA aims to create opportunities for each country to conduct its own cross-national analysis in order to enhance the understanding of the functioning of its educational system at all levels. The varying interests of participating countries contribute to the dilemma between desirability and feasibility (many stakeholders want an array of data, while there are practical limitations in collecting data in schools), and in these types of studies compromises have to be found among the interests of all participating countries. The IEA aims for a design and for instruments that are as equally fair as possible to all participating countries, while also allowing for national options. For example, in the 1999 TIMSS study in South Africa, where the majority of pupils receive instruction in a language other than the home language, a language proficiency test was included to allow for investigating relationships between language and achievement.

A final point that requires careful attention is the organizational and logistical complexities of the IEA studies. For instance, the 1995 TIMSS study involved the following: achievement testing in mathematics and science in forty-five countries; five grade levels (third, fourth, seventh, eighth, and final year of secondary school); more than half a million students; testing in more than thirty languages; more than 15,000 participating schools; nearly 1,000 open-ended questions, generating millions of student responses and performance assessments; questionnaires from students, teachers, and school principals containing about 1,500 questions; and many thousands of individuals to administer the tests and process the data.

Conclusions

IEA studies do not lead to easy answers to complex educational problems, but they contribute to the body of knowledge of how educational systems work and of optimal conditions for teaching and learning. An example can be found in a 1992 report by T. Neville Postlethwaite and Kenneth N. Ross, who determined on the basis of cross-national analysis of the IEA Progress in International Reading Literacy Study that a large number of variables (including school, teacher, teaching, and student variables) influenced reading achievement. Their analyses illustrate how IEA studies can contribute to informed decision-making by policymakers and create an awareness of the rich variety of educational settings and approaches around the world. On the other hand, the types of studies conducted by the IEA have some limitations and also receive criticism. Technical criticisms have largely been addressed in the recent studies, and critiques have become increasingly political.

Finally, reflecting on nearly half a century of IEA activities, a number of developments have occurred and benefits have emerged. IEA studies have moved beyond simply international comparative assessment scores and contextual information. They have contributed to the national and international education community in various ways. They have provided possibilities of addressing regional issues as part of an international comparative study, linking national assessments to international assessments, and for developing countries, in particular, the opportunity to collect baseline data on education. Important benefits of the international comparative IEA studies have been the development of education research capacities in many countries, culminating in the development of a network of researchers and specialists who can be drawn from by both governments and other agencies both nationally and internationally.