RYH Comments | CPS Draft Performance Policy | 8.5.13

CPS uses a Performance Policy Metric to assign schools a level rating and make decisions about probation and school actions. Attached is the latest draft of the revised CPS Performance Policy, which is likely to be voted on at the August Board meeting. We are meeting with CPS today to offer feedback (see comments below). Please share any thoughts that you have on this:

Raise Your Hand Comments on Draft CPS Performance Policy

Overall, the new formula proposed by CPS attempts to quantify and model some of the complex goals – worthy goals – of the new administration. However, after careful review, we caution that as it stands the current formula will misrepresent certain subgroups and populations, demonize others, and leave many children behind. We urge CPS to amend the policy to better recognize programs that demonstrate growth despite persistent challenges presented by factors such as poverty and high mobility rates; to incorporate evidence that emphasizes more long term outcomes to guide long term decisions, such as closing a school or changing leadership; to increase weight to the Five Essentials; and to give less weight to an aggregate measure of testing instruments designed to inform instructional strategies for individuals and small groups.  

The Five Essentials

The Five Essentials are a tremendous asset that is currently being underutilized. These are the research-based input that have been demonstrated to result in more effective programs. Making sure schools score well on the Five Essentials should be something that school leaders should strive towards, and we recommend making the Five Essentials a large factor in performance accountability so as to steer people/institutions in this direction. We estimate the 5E should be weighted at least twice what it is currently weighted. Thankfully, the district has already invested in collecting this data over time, so it should operationally be simple to include.

We cannot urge CPS strongly enough that we recommend a clear  emphasis on collaborative teaching, ambitious instruction, and involved teachers. In the current model these are either assumed as inputs or not part of the value formula.

Overemphasis on Aggregate Measurement of Tools Intended for Individual Learning

We concur that student growth is a laudable goal. NWEA MAP is a tool for  instructors to better understand where individual children at in order to strategize how to move them along a learning spectrum. Unfortunately, aggregating individual results can have unintended negative effects on certain classes of students and/or schools.

For example, while not intended to do so, and hopefully not the norm, data walls and aggregation of individual data are sometimes misused by teachers, administrations and district leadership in Chicago and beyond. The result is that children who are deemed “too hard” to move into the next category (e.g., up to “meets”) get ignored, as effort invested in those children are presumed to be less likely to yield a favorable metric. As this metric guides over half of the new Performance Policy, this is likely to have a chilling effect on students who are in the most need of help.

We are certain that CPS in no way intends to leave these children behind. Therefore, it is critical to look more closely at alternatives that better reflect both inputs and outputs to measure growth, and to do looking a trends over time.

For example, if we compare how schools or cohorts achieve with cohorts of students, we see significant growth in some schools that might not adequately be reflected in the current metric. While some would argue that we can ill afford to wait, we can actually already learn a lot more from longitudinal data than a yearly snapshot. A scan of the state report card for example, shows schools that serve children scoring relatively low (40-50% meets) in math and reading in third grade, and moves these children to 70-80% by eighth grade. Yet these schools and these children are judged against the same standard as selective enrollment schools such as (let’s look at Skinner North??) where scores tend to plateau. Yet schools that move children double digit points – children typically from at risk subgroups -- are therefore, effectively penalized by comparing them to the same standard as children who test in to selective enrollment programs and are exposed to an accelerated curriculum. This is akin to taking credit for a home run when the child (and her school) are already on third base.

There are additional reasons that we remain concerned about using short term metrics to guide long term decision making. For example, research on dual language programs indicates that children below average on standardized tests until after 4 to 6 years in an effective program, after which they outperform their monolingual peers. Yet how are these third and fourth grade classrooms evaluated with an overemphasis on NWEA/MAP? These grades occur before the threshold studied by the research (not to mention the fact that the majority of ELLs in such programs are Latino, and Latinos participate in early education at a rate of half of that of Caucasians and African Americans). Yet again those programs and their aggregate scores are measured by the same yardstick as the highest performing selective enrollment schools. In some neighborhoods, dual language programs are operationalized as heritage language programs because there are not enough students from English speaking homes attending. These programs build additively on the home language of the child, building literacy and fluency in both languages and bridging concepts between home language and English. Many of these programs also experience remarkable mobility from year to year or even within a year. The current formula overly penalizes these programs, which typically serve a high proportion of low-income, Latino, English language learners.  

And, even to a limited extent, some schools may see scores drop as parents engage in higher than normal amounts of churn or “shopping around” among programs, often gifted, classical and selective enrollment. For example, as children approach middle school, their parents angle to get them into Academic Center programs to avoid the wringer of the selective enrollment high school process. The current formula does not reflect the mobility of cohorts of children into accelerated programs in middle schools. 

Finally, highlighting performance change among special populations is a well intended goal, but in practice it can make schools resent these populations because “they bring scores down.”

Punishing Whole Schools For Individual Actions

The new policy requires a “Minimum Test Participation Rate of 95%." Clearly this is a response to the growing number of families nationwide who are questioning the return on investment in exposing their children to a continual battery of assessments. We urge CPS to abandon this requirement or simply exclude families who choose to opt out from this minimum participation requirement. Many parents and educators point to the decreased cost and advantages of using limited sampling to test children for academic achievement. In a district where high performing programs are losing vital programming such as music and art, RTI, world language and more due to unprecedented budget cuts – in a world where neighborhood schools which are anchors in vulnerable communities --  it is understandable that families are voting with their feet, and we’ve been told that CPS is looking into reducing the amount of testing. Until that effectively happens, the 95% Minimum Participation Rate sets up additional  and unnecessary conflict. Most families have yet to see the value in the battery of standardized tests their children endure. Demonstrate using valuable assessments, and 95% of families will comply.

Alternatives to Levels or Tiering Schools

We would also like to broach a conversation that explores a different way of modeling schools, shifting from Levels or Tiers to something more sophisticated and reflective of the inputs and outputs.

Data Point

Lastly, we assume there to be an error in the elementary column where NWEA Grades 3-6 are at 10% and NWEA Grade 2 is at 5%. We are concerned this means second grade scores are double counted over any other grade scores. Our assumption was that pulling out 2nd grade was to make them of less -- not more consequential.  If this is the case, you figure that each 3-6 grader 'contributes' 2.5% so the 2nd grade 'contribution' should be 1.25% if we want it as half.