What started as a subtle flaw in the has widened into a full-scale debate with national implications: Is too much riding on one fallible assessment?
The mistakes in the 2006 exam were discovered last month, as state education officials were looking at this year鈥檚 test results and puzzling over a sudden performance drop. In contrast to 2006, when the proportion of 3rd graders reading at grade level jumped from 67 percent to 75 percent, and previous years, when that percentage rose only 1 to 3 points annually, the scores for 2007 fell back by 6 percentage points to 69 percent.
The nosedive on the test itself was significant enough, but in Florida, any ripple in FCAT performance can send waves of consequences crashing through the state鈥檚 education system. High school graduation is tied to the test. So are the annual letter grades, A through F, with which schools get labeled. Teachers鈥� merit pay is also contingent upon the scores, as is student promotion from 3rd to 4th grade.
In no other state, according to assessment experts, does so much hinge on a single exam, though they say others are following Florida鈥檚 lead.
鈥淚t鈥檚 getting more and more common鈥攖he pressure鈥檚 ratcheting up nationwide,鈥� said Daniel M. Koretz, a professor of education at Harvard University and an associate director of the , based at the University of California, Los Angeles. 鈥淭he problem is that if something goes wrong, it makes it that much worse.鈥�
What went wrong in 2006 was invisible to the untrained eye, and even to the trained ones at San Antonio-based Harcourt Assessment鈥攖he company that created the FCAT. It also slipped by the Florida Department of Education, which has accepted responsibility for the error and is regrading roughly 204,000 exams.
According to education department spokesman Tom Butler, some 2006 鈥渁nchor questions鈥濃�攐nes whose difficulty must remain consistent from year to year, though the questions themselves are different鈥斺�渨ere probably easier for the kids than they should have been.鈥�
Compounding the problem, those questions were moved from the back of the exam, where they were in 2005, to the front, artificially increasing the chance that students would do better on them. 鈥淎 student is going to be fresher in the beginning,鈥� Mr. Butler said.
Inquiry鈥檚 Scope Widens
that 3rd graders who ought not have been promoted, based on their 2006 scores, would be given a pass. School grades will also stand, she said.
But it鈥檚 not yet known how the federal government will calculate schools鈥� adequate yearly progress under the No Child Left Behind Act, or whether other accountability measures indexed to the FCAT will be grandfathered in.
The FCAT Advisory Committee鈥攁 group of critics, parents, and others convened by Ms. Blomberg to pick experts to audit last year鈥檚 exam鈥攈as yet to recommend whether teachers might lose merit pay based on the 2006 results. But the panel has already widened its scope, and is now asking questions about the FCAT as a whole.
It鈥檚 far from alone. The , a nonprofit organization opposed to the FCAT, has written an open letter to Gov. Charlie Crist, a Republican, asking for a probe of all recent FCAT scores. The state鈥檚 House Democratic Caucus also wants the governor to investigate whether the promotion of 3rd graders broke any state statutes requiring that promotion be based on student proficiency.
State Sen. Don Gaetz, the chairman of his chamber鈥檚 pre-K-12 committee, also has launched an inquiry. But he cautioned those whom he called 鈥渃ritics of accountability鈥� not to throw the baby out with the bath water.
鈥淚f we found that less than 10 percent of X-rays in a hospital were being misread, we would not throw out all the X-ray machines in the building,鈥� the Republican lawmaker said. 鈥淲e would find out what went wrong and resolve that problem.鈥�
Contract Renewal Ahead
Some observers say part of the problem is the testing companies themselves.
鈥淭esting is a human endeavor, and humans make mistakes,鈥� said Robert Schaeffer, the public education director of the , or FairTest, a watchdog group based in Cambridge, Mass., that is opposed to misuse of standardized tests. But, he added, 鈥渉umans make more mistakes when they鈥檙e stretched thin, which is the case with the testing industry 鈥� since No Child Left Behind.鈥�
The 5-year-old federal in all states in grades 3-8 and once in high school, has created 鈥渁 real shortage of top-level testing professionals,鈥� Mr. Schaeffer said. 鈥淏ut also at the lower end of the employment chain, there is a problem getting sufficient folks to process and score tests.鈥�
He pointed to a recent rash of problems associated with state standardized tests, including a series of technical glitches that interrupted Virginia鈥檚 online Standards of Learning testing last month. Thousands of students will have to retake those exams.
Jay A. Diskey, the executive director of the Association of American Publishers鈥� School Division, a testing-industry group based in Washington, dismissed questions about test companies鈥� capacity.
鈥淚 see an industry that is performing well,鈥� he said. 鈥淭hat鈥檚 not to say there aren鈥檛 occasional issues. I鈥檓 not seeing the strain that some pundits may be seeing.鈥�
Harcourt Assessment鈥檚 contract with the Florida education department is up for renewal this year, and despite the problems with last year鈥檚 FCAT, Mr. Butler, the department spokesman, said the intent is to renew the contract.
鈥淸Testing companies] work hard, and they know what they鈥檙e doing,鈥� said David Miller, a professor of educational psychology at the University of Florida who writes often about assessment. 鈥淲ith the amount of testing going on in this country, the testing companies are straining. It doesn鈥檛 take much for problems to appear.鈥�
