A couple posts ago (before the posts on charter renewal and school diversity), I looked at SBAC results overall citywide and by subgroups. In this final post on the 2018-19 SBAC trends, I’ll be looking at school-level 5-year trends. To accompany this post, I created this Tableau dashboard that lets y’all dig deeper into individual school results. We also talked to representatives from OUSD, ACOE, Aspire, and Pivot Learning to get four perspectives on how key organizations are using this data to support differentiated assistance for schools.
Let’s look at how schools over the past 5 years in ELA versus Math:
Taking a page from Oakland Achieves and CA School Dashboard’s 5x5 grids, we can categorize school’s progress based on their combination of growth and proficiency levels. Each quadrant has its own set of challenges and therefore its own set of interventions needed. Ideally, you’d want all schools to be in the upper right-hand corner, where there is high growth and high proficiency rates.
Figure 2. Visual of the “Oakland Achieves” framework for categorizing progress
Figure 3. Comparing 5-year changes in proficiency rates versus final proficiency rates in 18-19 for Math and ELA. Link to Tableau.
Here’s the link to the Tableau dashboard (with plenty of filters, from subgroups to grade spans) for those who are data-curious.
With the majority of schools needing support of some kind (i.e. they’re not in the upper right-hand quadrant of doing well and improving…though these schools need their own type of support as well), we asked different Oakland education providers how they plan to support these schools towards improvement. This is usually called “Differentiated Assistance” or “DA” given that schools in different places need very different types of support. Here’s what they said:
“ACOE has really stepped up to take on the differentiated assistance process mandate. I appreciate Educate78 asking these questions. It’s important in each of the city ecosystems for support organizations to understand what it is we’re trying to do.
No Child Left Behind had a lot of sanctions, but as we’ve learned they don’t produce the type of results that we are looking for. By year 3, Program Improvement schools were putting together restructure plans, then bringing in new leadership in year 4, and by year 5 reconstituting yet again, and this unsuccessful process was happening primarily in Black and Brown communities.
We need a support system. That’s what it’s called in California. Whether it’s support to districts or now soon with charters. We need to be a support system – and that’s an incredible shift, an important nuance. We can’t always we be asking, “Where’s the hammer?”
It’s important to understand how new this is. With OUSD, we began our process with the Fall 2017 state dashboard data. So the District has really only been in the process for about a year and a half. We began by looking at all of their data. A team of experts from ACOE went in to look at all of OUSD’s data. We go through a process using the district – or Local Education Agency (LEA) – self-assessment tool whereby OUSD rates itself on multiple measures. The tool is used by CCSESA – multiple counties use it. The theory of action behind it is that districts do matter.
Charters have not been identified into the differentiated process yet. By statue, we will begin looking at them with the Fall 2019 release expected the week of December 9. There will surely be some identified for differentiated assistance. ACOE will first handle DA only for ACOE-authorized schools. ACOE does not have a role in providing DA for OUSD-authorized charters until three years from now. ACOE is actually the geo lead for the entire Bay Area, so after three years we will be handling DA for all charters that are not receiving DA from their local authorizers. It is going to be a challenge, but I am proud of my team for stepping up in this way.”
“Considering our significant opportunity gaps, we must have collective urgency about supporting all schools in getting on the right track.
This grid framework is really helpful in illustrating where we are seeing schools trending in the right direction (or not) with the percentage of students meeting standards on SBAC. In alignment with the state CA State Dashboard, we are also using the average students’ distance from standard (DFS) to monitor progress for schools and student groups. Our Instructional Focus plan names continuous growth for every student towards meeting or exceeding standards as our shared goal, and DFS helps us see progress for all students, including newcomers and students furthest from opportunity.
Looking at the data this way, we have seen some growth for elementary students (+14 DFS in ELA and +8 in Math) and minimal growth in MS last year, whereas high school performance unfortunately has decreased. We also look at cohort matched data using the CORE Growth Metric to understand a school’s impact on the students from where they started. This helps us get to a deeper level of differentiated assistance for all schools no matter where they are at on the grid.
To improve outcomes for all schools, we have launched a 3-year plan to provide quality, standards-based curriculum to all schools, foundational PD to all teachers, and coaching for school and teacher leaders to facilitative data-driven learning communities. An important component of this plan is our standards-based system of interim assessments and data reporting.
This year all students in grades 3-8 are taking three common SBAC interim assessments in Math and ELA (HS students take one) and students in K-1 are taking new foundational literacy assessments three times per year. At the end of each assessment cycle, we hold school and district data summits, at which we monitor progress towards end-of-year goals and determine where more supports are needed. This allows us to understand the practices that are producing growth and determine where interventions are needed before the end of the year.”
(Aspire is one of the largest charter-management organizations in our city, and is actually based in Oakland, with schools across Northern California and beyond)
“When looking at differentiated assistance, we start with data. We look at data absolutely, all the time. If a school is consistently not achieving at proficiency and declining, we probe and ask questions:
We provide intensive coaching and resource support to grow, make changes, or shift, PD, etc. If, ultimately, however, things don’t change, then we ultimately want to do what’s right by children not by the school or even the staff.
When schools are not working, we have to make hard decisions. We are merging our East Palo Alto schools – phasing out our middle school and moving to a K-8 model. For high quality schools, we have a professional development monthly, instructional rounds, and learning walks where we identify and highlight a best practice. Then we leverage that expertise in monthly professional development meetings.
When we see declines, we really take the time to analyze – specific, strategic, and targeted support to address the problem. At Cal Prep in Richmond last year, our grad rate was 83/84% – not good enough for our standards – so we asked: what are we trying to solve and how do we solve that problem? This year – a 94% graduation rate with only one student who did not graduate because they shifted to a private school.
The last thing I would want to say about differentiated assistance and in general with educational improvement: Retention really matters – when you retain dedicated, passionate leaders, you can really make an impact and a change.”
(Pivot Learning is an Oakland-based educational consulting group that assists districts, charters, and other public education institutions across the country)
“Evaluating the context of the school, district, or organization on the front end is the most important step to effective differentiated assistance.
The data only tell you so much when you look at student achievement. The question is what’s driving those issues of student achievement?
The most fundamental way to determine that is to look at what’s going on with teaching and learning – what are the underpinnings organizationally around that. High teacher turnover? Teacher absences? New teachers? Where are teachers – and principals and other leaders inside the school – in terms of their development and alignment?
And there are some organizations that face much more basic challenges. I won’t name names, but we work with at least one district that’s faced state takeover. We’re working on very technical issues there. Stuff everybody else has already addressed. Just to be functional. They can’t do basic stuff like complete an IEP on-time. That’s one level of assistance. We need to get them a policies and procedures manual. If you start working on more complex things like classroom walkthroughs when you don’t have the technical stuff done, then you’re wasting your time.
If the systems are more higher functioning already, then you look at higher functioning levers:
All of these are arranged along a continuum, and must be aligned. You can select a really good curriculum, but if teachers don’t use it, it’s useless. It’s a nuanced understanding within each organization. The context is deeply important.
Lastly, how many things are you trying to do all at once. Larger organizations, more often districts, try to do it all at once. Smaller organizations like charters tend to do it a little better by focusing on only three or four things. Even if you have a weak curriculum but implement it really well you might get better results than if you have a great curriculum and implement it very poorly.”
In this analysis and others so far in SY19-20, we’ve been looking at SBAC proficiency data, which is great in helping give a quick pulse check in how many kids are meeting the standards, but it has its limitations. There are two big flaws with relying solely on SBAC results for getting pulse check on how a school is serving their students:
Fortunately for us, the CA School Dashboard addresses (though not perfectly) these two flaws with its multiple indicator approach and emphasis on both growth AND performance. I am BEYOND excited that the dashboard data will be released this coming Monday, December 9 (or so the CDE has advertised). I can’t wait to see how Oakland schools have progressed over the past year. Stay tuned!
My name is Carrie Chan, and I’m Educate78’s data analyst (aka resident data nerd and cruncher). As a former OUSD student, I care a lot about Oakland public schools. This blog series, “Crunched!”, takes a data-driven approach to important, relevant questions facing Oakland public schools, sharing out easily digestible data takeaways. Please email me with ideas, requests, or feedback.