International Assessments

The Third International Mathematics and Science Study (TIMSS) is the largest and most ambitious educational assessment ever done under the auspices of the International Association for the Evaluation of Educational Achievement (IEA). The TIMSS data collection in 1995 involved testing more than a half-million students in more than forty educational systems (usually countries) around the world. Students were assessed at five different grade levels, and students, as well as their teachers and principals, were given questionnaires about their backgrounds, attitudes, and practices. The TIMSS data collection in 1999 focused on eighth-grade students in thirty-eight countries.

The TIMSS study resulted in many provocative findings, which are published in the TIMSS international reports. The results showed that the averages of students of participating Asian nations (Hong Kong, Japan, Korea, and Singapore) were higher than those of students of other nations in mathematics at both the elementary and middle school levels. Japan and Korea also did very well in science at these levels, although Australia, Austria, and the United States also performed highly at the elementary level, and the Czech Republic performed well at the middle school level.

Testing was also done at the end of secondary school, where a sample of the total population of students was assessed in mathematical and scientific literacy. In addition, samples of students taking advanced mathematics or physics courses were tested on those subjects. The Asian countries were not among the twenty countries that participated in this assessment. The average scores of the Netherlands, Sweden, Iceland, Norway, and Switzerland were highest in mathematical and scientific literacy; the average achievement in advanced mathematics was highest in France and Russia; and the average physics test scores were highest in Norway, Sweden, and Russia.

The TIMSS results have been widely reported in the press and in numerous public reports. All TIMSS reports, including those cited above, are available on the Internet. TIMSS's website also contains technical reports that contain the details of the TIMSS methodology, and the raw TIMSS data are available at this site for those who would like to use the data to investigate different educational questions or research methodologies.

The study was administered by the International Study Center at Boston College and by the International Coordinating Centre at the University of British Columbia. TIMSS has been funded by the participating countries along with major contributions by the Government of Canada, the National Science Foundation (U.S.), and National Center for Education Statistics (U.S.).

The Aim of TIMSS

The TIMSS was established to improve the teaching and learning of mathematics and science in school systems around the world through a comparison of the curricula and practices of different countries, and to relate this information to the performance of their students. The research questions included not only what students in the participating countries had learned, but also how the curricula varied in different countries and what facilities and opportunities were made available for students to learn what was in their curricula. The relationships of students' performance to their curricula, educational opportunities, and backgrounds were also to be investigated.

IEA Studies of Mathematics and Science

The IEA has been involved in comparing the educational systems of various countries for many years. Four previous IEA studies of mathematics or science led up to TIMSS:

• First International Mathematics Study (FIMS), 1959–1960.
• First International Science Study (FISS), 1970.
• Second International Mathematics Study (SIMS), 1980–1982.
• Second International Science Study (SISS), 1982–1986.

TIMSS, which included both mathematics and science, was conducted in the Northern Hemisphere in 1995 and in the Southern Hemisphere in both 1994 and 1995. A second round of TIMSS, involving only eighth-grade students, was conducted in 1999. A third assessment is planned for 2003.

TIMSS Design

The design of TIMSS grew out of discussions in the late 1980s by many researchers who were involved in SIMS, and these discussions led to the Study of Mathematics and Science Opportunity (SMSO), which explored the curricula and teaching practices of a few countries around the world and initiated the development of the tests and questionnaires for TIMSS. The final design and its instrumentation were developed and approved by the participating countries in conjunction with mathematics and science education specialists and specialists in educational assessment.

Populations and Sampling

TIMSS defined three populations of students for assessment:

• Population 1–all students enrolled in the two adjacent grades that contain the largest proportion of students nine years old at the time of testing. In most participating countries, grades three and four fit this definition.
• Population 2–all students enrolled in the two adjacent grades that contain the largest proportion of students thirteen years old at the time of testing. In most participating countries, grades seven and eight fit this definition.
• Population 3–all students in their final year of secondary education, including students in vocational education programs. In most countries, this was grade twelve. Population 3 included two subpopulations: students taking advanced courses in mathematics and students taking advanced courses in physics.

In Populations 1 and 2, an early decision was made to sample intact mathematics classrooms so that information about teachers and students could be matched and studied. The students who were not enrolled in any mathematics class were treated as a separate classroom, and thus they could be selected for the sample. In Population 3, students were not sampled by classroom, but were classified according to their mathematics and science courses, and then individually sampled for assessment. All participating countries were required to assess Population 2, but assessments of Populations 1 or 3 were optional.

TIMSS Tests and Questionnaires

The TIMSS tests were constructed using mathematics and science frameworks that were agreed upon by the participating countries and subject-matter specialists. TIMSS included multiple-choice, short answer, extended response, and performance items. Countries were not required to administer the performance items.

In order to widen the curriculum coverage of TIMSS, a form of matrix sampling was used in which students in a population received different test items, except for a few items that were common to all booklets. In Populations 1 and 2, there were eight different booklets of items administered along with student questionnaires. Teachers and school questionnaires were also administered. In Population 3, nine test booklets were administered, with the particular booklet to be used dependant on the courses in which a student was enrolled. The teacher questionnaires were omitted because intact classrooms were not sampled. The booklets in Populations 1 and 2 required about an hour of student time, whereas the booklets for Population 3 required about ninety minutes.

Administration and Quality Monitoring

TIMSS was administered by personnel from the participating countries, and the TIMSS administrators were given extensive training to assure the high quality and comparability of the TIMSS data. International quality-control monitors were hired and trained to visit the national research centers and to review their procedures. The translations were also checked centrally to detect and avoid differences in the presentation of assessment questions.

Analysis and Reporting

The basic TIMSS database was constructed in the participating countries and then given extensive statistical scrutiny at the IEA Data Processing Center in Germany. Any unusual occurrences in the database were noted and adjudicated with the participating countries. The data were then sent to Statistics Canada for a review of the sampling and construction of sampling weights. The data also went to the Australian Council for Educational Research for scale development. The scaling was done using a variation of the Rasch model. The scaled database then went to the International Study Center at Boston College for analysis and reporting.

TIMSS was a very complex study that required the cooperation of the many countries and contracting organizations involved. Cooperation was critical to assure that the tests were appropriate for all countries and that the administration of TIMSS was uniform. The coordination required many meetings in which strategy and tactics were discussed and decided. Many training sessions were required to assure that all phases of the TIMSS were carried out successfully. The flow of data from the participating countries to Germany, then Canada, Australia, and finally to the United States required careful monitoring. Finally, the final reports were designed and approved by the participating countries before the data were available. The details of the procedures are given in the TIMSS technical reports.