The majority of Hawaii public school teachers — 98 percent — were deemed highly effective or effective educators, and fewer teachers received marginal ratings for the 2014-15 school year, marking the second year of ratings on a high-stakes evaluation system that ties performance to pay raises, tenure and termination.

The high ratings for most of the teaching force mirror results from the previous school year, when 98 percent of teachers received “effective” or “highly effective” ratings — the highest of four categories — and have prompted concerns from some Board of Education members that the methodology may not be rigorous enough and might be producing “false returns.”

Under what’s known as the Educator Effectiveness System, or EES, half of a classroom teacher’s annual rating is based on student learning and growth, measured by standardized test scores and data-driven academic goals. The other half is based on teacher practice, measured through classroom observations and student surveys.

Only teachers rated as effective or highly effective are eligible for collectively bargained pay increases in the year after the evaluation. Marginal teachers are given an opportunity to improve, while an “unsatisfactory” rating is cause for termination.

Evaluations questioned

For the 2014-15 school year, 4,206 teachers were deemed highly effective, representing 35 percent of teachers rated, according to results released last week by the state Department of Education. Some 7,478 teachers were rated effective, representing 63 percent of teachers rated.

“That’s a high percentage, which is great, but I’m just wondering what does this reveal about the evaluation system itself — do you think it’s rigorous?” BOE member Patricia Halagao asked DOE officials at a recent meeting of the board’s Human Resources Committee.

“I think it’s a reflection of how good our teachers are in the classroom, and I really do believe that the vast majority of our teachers do a great job,” schools Deputy Superintendent Stephen Schatz said. “To what degree this is an accurate representation of the workforce and how we’re doing, I think that’s a question that we continually wrestle with.”

He added that the system is still a work in progress and that the department regularly receives feedback on its design and implementation from teacher and principal groups.

Some board members questioned the rating methodology, given that the matrix used to determine an overall rating allows a teacher who gets a marginal score in one of the two areas measured — student growth and learning or teacher practice — to still receive an “overall effective” rating.

“I have serious questions about the effectiveness of those teachers. I don’t think it’s asking too much to be at least effective in one of the two areas,” BOE member Jim Williams said. “You can go through all of the 50 percent (student) growth and the 50 percent teacher practice, and the observations and the portfolios, and that can be very rigorous, but in terms of ratings, rigor is really reflected in this matrix. … You can have a rigorous system, but if your matrix is not rigorous, then I think we will get some false returns.”

In the lower categories, 69 teachers were marked as marginal — a significant decrease from the 233 teachers who were rated marginal in the previous school year. Meanwhile, 18 teachers were found unsatisfactory, compared with 24 teachers the year prior. Those 18 teachers have either left the DOE or are challenging the rating through their union, Barbara Krieg, assistant superintendent for human resources, told the committee.

Testing criticized

Corey Rosenlee, president of the Hawaii State Teachers Association, said the results validate the work of Hawaii teachers, but he takes issue with the idea of tying teacher pay to standardized test scores, which, he argues, teachers have little control over. He said teachers support evaluations but favor a model that is “collaborative versus evaluative” and fosters positive interaction between teachers and administrators.

“Teaching cannot be boiled down to a number. Teaching is an art,” Rosenlee said in an interview. “For example, if you want to try to measure teaching by students’ test scores, what research shows is that test scores are highly correlated with socioeconomic status and not necessarily a teacher’s ability. They use student growth when the American Statistical Association has said that that’s an invalid measurement because it fluctuates so heavily … and it also penalizes special-education and (English Language Learners) teachers.”

He added that using test scores often leads to an overemphasis of tested subjects. “As long as teachers believe that student test scores are being tied to their evaluations, then they start teaching to the test, and we deprive our children of a well-rounded education,” he said.

Hawaii is among 18 states rated as having a strong policy for requiring that student achievement be the “preponderant criterion” for teacher evaluations, according to the National Council on Teacher Quality.

Overhauling teacher evaluations was a key commitment in the state’s application for its $75 million federal Race to the Top grant, which Hawaii won in 2010 after pledging education reforms, including plans to turn around its lowest-performing schools, boost student achievement and improve teacher and principal effectiveness. The commitment to adopt evaluations that rate teachers on student learning and teacher practice was subsequently folded into BOE policy and the board-department joint strategic plan.

Under the state’s former Professional Evaluation Program for Teachers, or PEP-T, teachers were not evaluated annually and only classroom observations were used.

“The default was that every five years our teachers were evaluated,” said Schatz, the deputy superintendent. “The highest you could be was satisfactory, (and) the tool itself didn’t reflect any values or any clear expectations for the field.” He said the EES, by contrast, was designed to do two things: raise student achievement and enhance teacher effectiveness.

Workload ‘untenable’

The HSTA, in its 2013-2017 labor contract, agreed to annual performance-based evaluations. But the statewide rollout two falls ago proved controversial among teachers who said they didn’t understand its complex design and were overwhelmed with the work required to prepare for the areas measured by the evaluations. Principals also decried the initial implementation, saying the evaluations were extremely time-consuming and negatively affecting morale.

“In our first year of implementation, we had a really heavy lift for our teachers and principals,” Schatz said. “We expected numerous observations every year for every single teacher, and it became pretty clear pretty quickly that we weren’t able to keep up with the workload and it was a burden on both teachers and principals that was kind of untenable.”

The DOE last summer announced more than a dozen changes to essentially cut in half the workload required to prepare for and perform the annual reviews. Schatz said annual surveys of teachers, jointly commissioned by the DOE and HSTA and administered by Ward Research, show more educators have a better overall understanding of the EES and its individual measures and received better training this year compared with last year.

“Quite honestly, in the first year it showed that there was a lot of confusion and there was a lot of frustration, and we committed to taking some serious actions,” he said. “In this year’s survey we saw pretty dramatic improvement both in understanding of the components but also some pretty significant changes in teachers’ perceptions of the value of the components.”

Kauai High School English teacher Jonathon Medeiros, who received a “highly effective” EES rating, said he believes the evaluation is an improvement over the old system.

“I feel like the intent of the EES system is to provide a space for teachers to reflect on their own practice based on all of these different data points to hopefully become a better teacher, adjust and grow,” he said in an interview. “So in that sense I feel like it’s a positive step forward. If a teacher wants it to be useful, it can be very useful, and it was for me.”