Monday, December 9, 2013
More than three-quarters of Maine's eighth-graders performed below standard on the state writing test for 2007-08, prompting education officials to toss the results and try to figure out why so many students missed the mark.
State Education Commissioner Susan Gendron and her staff say the one-question test was somehow flawed because 78 percent of the estimated 14,900 eighth-graders who took the exam failed to write a persuasive essay as required.
That's a 50 percent increase, over 2006-07 in the number of eighth-graders who failed to meet or only partially met state writing standards.
In a rare move, Maine's Department of Education found the test results inconclusive, and withheld them from school districts and the media when it released the latest Maine Educational Assessment scores in July.
The department's decision surprised even longtime educators like Tom Lafavore, director of educational planning in Portland Public Schools, Maine's largest district.
''I've never seen test results pulled like this,'' Lafavore said.
The department provided overall Grade 8 writing results to the Portland Press Herald/Maine Sunday Telegram only after the newspaper requested the information under Maine's Freedom of Access Act.
''It is our responsibility to ensure the validity of test data,'' Gendron said. ''It would be irresponsible for us to release data if that performance is based on a question that was unreliable.''
Gendron and her staff say parents shouldn't worry. Students are learning to write. The test triggered false results. Still, they say, they don't know exactly why.
The 45- to 70-minute test, administered last March, asked students to support or refute the following statement, known as a prompt: ''Television may have a negative impact on learning.''
Instructions outlined how the essay would be scored and listed 20 writing skills students should demonstrate, from identifying a logical position to using correct punctuation. The test included two lists of facts, pro and con, to use in the essay.
''Kids got ticked off at the (question),'' Gendron said. ''In many cases, it was an emotional response rather than the intellectual exercise we were seeking, so it was not an accurate reflection of their writing skills.''
One student's essay began: ''These facts are lies. I do my homework and get good grades even though I watch TV.'' This example came from an e-mail to Susan Smith, Maine's MEA coordinator, from Julie-ann Edwards, a staff member at Measured Progress, the state's testing consultant based in Dover, N.H. The newspaper obtained the e-mail through its request for records and internal communications related to the test question.
''This year, students often took issue with the prompt and fact sheet,'' Edwards wrote. ''They reacted emotionally, spouted a bit, and did not use the fact sheet information to support their argument.''
Edwards noted that eighth-graders who took the writing test in 2007 were able to draw from their own experience to sustain arguments for or against the following statement: ''Rather than maintaining separate teams for boys' and girls' sports, a high school is considering combining teams and having a completely coed sports program.''
''That did not appear to be the case this year,'' Edwards wrote.
Patricia Ross, spokeswoman for Measured Progress, referred questions to state officials.
Overall, less than 23 percent of eighth-graders who took the test last spring met or exceeded state writing standards, down from 48 percent in 2006-07, indicated a report from Measured Progress. That's a 52 percent decrease.
The marked difference surprised education officials because the television prompt had done well when it was field tested in 2005-06, Smith said.
The state started administering writing tests as part of the MEA in 2006-07. The MEA assesses reading and math skills in grades 3 through 8 as required under the federal No Child Left Behind Act. Maine tests writing skills and science knowledge in grades 5 and 8; federal law requires the science test but not the writing one.
Data on the frequency of test-question failure was unavailable. The U.S. Department of Education doesn't review the tests or individual questions states use.
''States do have to demonstrate the overall technical quality of their assessments, including the validity and reliability of the scores,'' said spokeswoman Jo Ann Webb.
She said another state dropped a question from its writing assessment because a newspaper had published the question before students took the test.
Measured Progress develops the MEA, which is reviewed by Smith's staff and teacher panels to make sure questions reflect standards outlined in Maine Learning Results, a teaching and testing guide established by the State Board of Education.
Updated last year, the guide sets learning targets for annual student progress in eight subject areas related to communication and problem-solving. The guide stipulates that by sixth grade, students should be able to write academic, persuasive essays like the one required on the eighth-grade writing test.
Smith said students also are familiar with the detailed writing instructions that accompany the test. She said the state encourages teachers to use writing samples, scoring guides and a similar list of desired writing skills in daily writing lessons.
Lafavore, the administrator with broad experience in developing and scoring academic assessments, noted that the structure of the writing test is typical, if not necessarily kid-friendly. He says the problem with the prompt may have been the subject matter and the premise that television may hurt learning.
''If I were a kid, knowing what I know about the influence of technology on their lives, my first reaction would be to completely disagree with the prompt,'' Lafavore said.
Regardless of why the prompt failed, Lafavore said he's glad the state didn't release individual Grade 8 writing results to districts and students: ''There just aren't that many kids in the state who would perform below standard if the test were valid.''
Smith said the department got a lot of feedback from teachers and administrators about the writing test. As a result, the structure and format will be adjusted.
As well, several new prompts that stress the use of analytical and argumentative skills were field tested last spring. Smith said she hopes the prompt used in 2008-09 will allow Maine's eighth-graders to demonstrate their best writing skills.
''That was our goal the first time and that is our goal this time,'' she said.
Staff Writer Kelley Bouchard can be contacted at 791-6328 or at: