This is the transcript of a speech made by Paul Ramsey, Senior Vice-President, Global Division, ETS, at a Breakfast Salon organised by ETS Europe - UK.

Introduction to ETS

Ian Lucas, who leads ETS Europe UK, and I want to welcome and thank you for joining us for what I hope will be a stimulating morning discussion.  My name is Paul Ramsey; I’m the senior vice president of the Global Division of Educational Testing Service (ETS). 

It may be that some of you don’t know much about ETS as until fairly recently it was a US-based-only organization.  ETS and its subsidiaries have offices and/or staff in 14 countries outside the US. (Canada, China, France, Germany, India, Jordan, Korea, Poland, Puerto Rico, Qatar, Singapore, Spain, Netherlands, United Kingdom).  We will soon have offices in Japan and Latin America.  ETS is a not-for-profit research organization that for more than 50 years has worked to help people pursue their dreams by providing them with assessments and products and services that accurately measure what they know and can do.

We try to help policymakers, educational authorities, parents, and teachers by providing and analyzing data so they can know what is and isn’t working with their students.  We perform macro-level research related to accountability and standards, as well as micro-research on classroom practices and individual students.

ETS is becoming a global organization and is learning from and sharing its expertise with educational institutions, businesses, and governments around the world to provide data-driven educational solutions to the challenges facing educators worldwide.

In my work as senior vice president of the Global Division of ETS, I’ve had the opportunity to travel around the world to try to learn firsthand about the global educational landscape.  I have the opportunity to meet with governmental and educational officials and practitioners world-wide, and I have noticed that in both developed and developing countries the questions among those who really care about students are the same:  “How does what we are doing and/or proposing benefit our children?”  “Is it efficacious?”  “How will it ultimately contribute to the well-being of our children and thus our nation?”

Educational Reform

I’d like to spend a few moments sharing some thoughts with you on what is taking place in primary and secondary education in the United States, as I think there are some interesting parallels with the educational issues the United Kingdom is grappling with.

Over the past decade and a half, the United States has been actively and passionately engaged in an effort to reform its educational system.  The real effort began with the vigorous push towards standards-based education.  More recently a new dimension has been added to the standards-based model:  an addition that entails monitoring students’ progress so that we know what is and isn’t working and can react accordingly.  This so-called new dimension has been christened “test-based accountability,” but, as I implied previously, this approach to monitoring learning is one ETS Research has fostered since its inception in 1947.  What has made this approach seem new in the eyes of US educators is its codification in the No Child Left Behind (NCLB) federal legislation.  This law applies to all 50 states and is designed to hold teachers, students, schools, and states accountable for student performance.  In some ways this legislation is akin to the education component of your Every Child Matters (ECM) legislation, though, if I understand it correctly, ECM is intended to be much more comprehensive than the educational purview of NCLB.  NCLB mandates that each state set learning standards, and then holds schools accountable for their students’ progress vis-à-vis these standards.  To measure and track student academic progress, the law requires annual assessments.

In very general terms, the law requires that each of the 50 states adopt actionable plans for ensuring that all their students meet high academic standards.  The federal government must approve these state plans and students’ academic progress is measured by their scores on standardized tests given annually for age groups 8-14 in reading and mathematics.  In addition, science assessments must be developed and put into place by the 2007-08 school year, and each student must be tested at least 3 times during primary and secondary education in science.

Let me say a few words about some of the features of NCLB that may be of particular interest to you.

• States are expected to raise student performance goals so that by 2014 all students, and all subgroups of students, will be performing at a pre-determined proficiency level.  Each year schools and all subgroups of students in those schools must make “Adequate Yearly Progress” towards this proficiency level.
• States must report test scores not only for individual schools but also for each subgroup of students within each school, e.g., low-income students, those belonging to racial or ethnic minorities, students with disabilities, students with limited English proficiency, etc.
• At least 95 percent of students in a school and 95 percent of each subgroup must participate in each assessment or the school will be considered not to have made Adequate Yearly Progress.
• If schools or the subgroups in a school do not make the so-called Adequate Yearly Progress for two years in a row, the school will be considered to be “in need of improvement”.  Federal funds could then be cut, and the school could be restructured or taken over by the state or by a private educational firm.

As you can see, there is some similarity with what happens here in England in that there is a heavy emphasis on testing to measure the extent to which students are or are not mastering the content and performance standards set for them, as well as to sanction those schools that are unsuccessful in assuring that their students do, indeed, master the standards.

The Importance of Well-Constructed Tests

Given how important assessments are to this reform model, it’s necessary that they provide an accurate, valid, and reliable measure of what students know and can do, but to do this they must be aligned with the curriculum and instruction.  One of the major projects my division, the Global Division, has taken on recently is working with a Middle Eastern government and with both in- and ex-patriot educators to build a 1-12 educational system from scratch which is “perfectly” aligned.

Assessments must be aligned with the content standards that specify what students should know and be able to do.  If there is misalignment here, the test cannot be a valid measure for standards-based accountability.  If you’re teaching one thing but testing another, who knows what the students know in such a misaligned system?

Even if the test is closely aligned to the content standards, performance on the test could be meaningless if the curriculum and instruction do not provide students the opportunity to meet those standards, i.e., the Opportunity to Learn (OTL).  It is, therefore, essential for curriculum and instruction to be aligned with the content standards and the test.  Students cannot be expected to do well on a test that measures content they have not been taught.

Tests, as I’ve indicated, are a necessary component of standards-based education reform, but they are not sufficient.  Tests by themselves can no more improve learning than thermometers can cure patients.

• There must be structures in place to provide teachers with Professional Development in appropriate methods for standards-based teaching.
• There should be transparency or adequate notice given to students and their parents that new standards for promotion and graduation are being put in place, and there should be a clear articulation of what those standards are.
• There should be, as I said above, an adequate opportunity for students to learn the material that will be tested and an opportunity to become familiar with the test format that will be used. 
• When the testing is over and the results are in, there should be meaningful learning experiences for students who performed poorly so they can improve, i.e., they should be given other opportunities to learn.  Students who performed poorly initially should be allowed to take comparable forms of the test, so they can demonstrate that they have learned.
• Test results should be examined to determine if changes need to be made in curriculum, instruction materials, teaching methods, and/or the tests themselves so as to ensure that these seminal components remain relevant and aligned with each other.
• There also should be a reporting system for the test results that is understandable and meaningful to the public and actionable by teachers, i.e., teachers should be able to determine what they need to do or teach in light of these results.
• It is also very important to look at the consequences – both intended and unintended – of testing.  Testing will and should change instruction, but it will change it for the better only if we have constructed tests worth teaching to.  Also, we need to look at what is happening in those content areas that are not being tested.  Are they receiving the appropriate emphasis?  Too, we also need to look at the school-leaving rate.  We need to understand whether enforcing higher standards is pressuring more students to leave school – a possible detrimental effect of testing – or whether improved teaching associated with articulated standards is making it easier for students to succeed and thereby encouraging them to stay in school and complete their education.

The Goal:  Better Instruction

Having test results that give an idea of what students know and can do on an individual, class, school, or district level is a good, but if you cannot translate this good into better learning opportunities for students, what difference does it really make?  If the tests don’t lead to opportunities to learn, you end of with testing, not for learning’s, but for testing’s sake.  A critic of too much testing once said, “Weighing a pig often doesn’t make it fatter.”

One thing ETS does is help schools to understand data better and supports them in finding ways to use the data to drive instructional practices that can improve student achievement.  Instead of developing a learning management system that is basically an administrative tool, we’ve developed one with the goal of promoting individual and classroom learning.  Teachers and headmasters are better able to improve their instruction and curriculum if they are given the tools to use data intelligently to align curriculum, instruction and assessments, and to discern trends in what their students do and don’t understand.

The heart of this learning system is a calibrated item bank with questions that teachers can use to build interim assessments that indicate whether students are mastering what they are being taught, e.g., whether they are achieving benchmark levels the school has set and how quickly they are progressing.  The tests provide immediate information on what students are and aren’t understanding.  Teachers can analyze interim assessment results by school, grade, or sub-groups vis-à-vis individual questions or groups of questions.  If linked to national test results, schools can determine how near or far they are from identified performance targets.  The tool also allows schools to give parents secured access so they can see how well their child is doing in general or on each standard in particular.

A Caution

Let me end with a caution about testing this audience hardly needs to hear.  A test score is like a snapshot.  It is static, capturing a moment in time within a fixed context.  A knowledgeable eye analyzing a photograph can evoke thoughts about how the picture might have turned out differently with, say, a slight change in lighting.  Analyzing the results of tests in the context of other factors can lead educators to embark on new directions to improve student learning, classroom instruction, and the school environment.

Too often, however, hasty, uninformed judgments about test results lead to superficial responses to complex educational issues, rather than to a meaningful, truly educational response to the scores.  Rather than a guide, the tests become a cudgel.  Educators need to interpret test scores carefully, determine how to use them to provide better learning opportunities for their students, and consider them in assessing changes that are needed in curriculum, instruction, and the tests themselves. 

It is important always to remember that while tests are a very important part of the educational picture or puzzle, they are only one part of a very complex, multi-dimensional picture that must be interpreted by knowledgeable educators, and even by them with great care.

Top

Print