Donate
Independent, objective, nonpartisan research

R 0417IUR

Authors

R 0417IUR

Tagged with:

Publication PDFs

Database

This is the content currently stored in the post and postmeta tables.

View live version

object(Timber\Post)#3741 (45) { ["ImageClass"]=> string(12) "Timber\Image" ["PostClass"]=> string(11) "Timber\Post" ["TermClass"]=> string(11) "Timber\Term" ["object_type"]=> string(4) "post" ["custom"]=> array(6) { ["_wp_attached_file"]=> string(13) "R_0417IUR.pdf" ["wpmf_size"]=> string(6) "469773" ["wpmf_filetype"]=> string(3) "pdf" ["_edit_lock"]=> string(12) "1495273592:1" ["wpmf_order"]=> string(1) "0" ["searchwp_content"]=> string(44910) "APRIL 2017 Iwunze Ugo and Laura Hill Supported with funding from the Evelyn and Walter Haas, Jr. Fund Student Achievement and Growth on California’s K –12 Assessments © 2017 Public Policy Institute of California PPIC is a public charity. It does not take or support positions on any ballot measures or on any local, state, or federal legislation, nor does it endorse, support, or oppose any political parties or candidates for public office. Short sections of text, not to exceed three paragraphs, may be quoted without written permission provided that full attribution is given to the source. Research publications reflect the views of the authors and do not necessarily reflect the views of our funders or of the staff, officers, advisory councils, or board of directors of the Public Policy Institute of California. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 3 Introduction 5 More Students Are Meeting Achievement Standards, but Gaps Persist 5 Test Score Growth Is Uneven 9 The New State Performance Measure Combines Achievement and Growth 13 Conclusion 16 References 17 About the Authors 18 Acknowledgments 18 Technical appendices to this paper are available on the PPIC website. California’s recent emphasis on local control for public K –12 schools recognizes that the work of improving student performance is largely the responsibility of school districts. The Local Control Funding Formula (LCFF) and the Local Control and Accountabi lity Plans (LCAPs) provide the resources and structure to help districts meet their goals, with a particular focus on closing achievement gaps. Yet in the first few years, LCAPs have been criticized for having poorly articulated goals, especially for high -need students. Now that two years of data from California’s new standardized test are available, we are in a better position to evaluate early implementation of both the Common Core State Standards and the new Local Control Funding Formula, particularly its impact on economically disadvantaged students and English Learners. We find that California’s school children did much better in the second year of the new statewide standardized tests, though achievement gaps have not substantially narrowed.  About 49 percent of students met grade -level achievement standards in English Language Arts, while 37 percent met the standard in Mathematics. Compared to the other large states using the same test, those shares were somewhat lower, but California’s incre ases from the year before were nearly twice as large.  Economically disadvantaged students and English Learners also largely showed improvement in the shares of students meeting the standards — although those increases were not at rates fast enough to close substantial achievement gaps with students who are not disadvantaged.  High -need districts (where over 55 percent of students are economically disadvantaged or English Learners) saw lower levels of achievement on average, with about 33 percent of students meeting the standards compared to about 60 percent for districts with fewer high -need students.  While most districts saw similar growth regardless of their share of disadvantaged students, districts with low levels of achievement and growth tended to have higher shares of high-need students. These results are especially troubling because they indicate that disadvantaged students are falling further behind. Another way to understand school and district performance is to compare performance after adjusting f or student demographics. We find that the districts and schools that either exceeded or failed to meet expectations according to estimates based on their student demographics often differed from those identified as outliers by the state’s new academic acco untability measure. We also find CONTENTS SUMMARY PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 4 that some schools have exceeded or failed to meet expectations repeatedly over the past several years. Finally, we find that results at many schools outperformed or —more often —lagged behind the overall results in their dist ricts. This report can help districts assess student performance—overall and for high- need students in particular— compared to other districts with similar demographics. The districts and schools with better -than- expected performance on the Smarter Balance d tests can be a valuable resource to the districts and schools that are still struggling to implement the new state standards and adapt to their new responsibilities to improve their own accountability plans. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 5 Introduction The second year of California’s statewide administration of the Smarter Balanced1 assessment allows parents, educators, and policymakers a second look at achievement and a first look at growth for K–12 students as measure d by this standardized test . Test results are a useful tool for assessing the early implementation of both the new C ommon C ore standards and district Local Control and Accountability Plans (LCAPs) . The good news is that most students are making progress. I n order to address long-standing achievement gaps, districts receive extra funding based on the share of their students who are high -need (defined as economically disadvantaged, English Learner or foster youth) through the new Local Control Funding Formula (LCFF) . For this and other reasons, we are particularly interested in understanding the progress that these high -need students are making. Note that s tandardized assessments are only one way to measure student success, albeit an important one. The State B oard of Education is developing a new accountability system with multiple measures, including academic achievement and growth, as well as social and emotional learning. When we examined the first -year Smarter Balanced results, we found that achievement —the share of students meeting the state’s academic standards—was lower than it was in the final year of the old statewide assessments, the California Standards Test or CST (Hill and Ugo 2016) . This was not unexpected given the new test content and design, as well as the new standards. We also found that high -need students had some of the lowest scores, and the achievement gaps between those students and others who are not disadvantaged were larger than they were on the CST. Using the second year assessment, districts and schools can measure both achievement and growth . In this report , we first describe how California’s students performed on the second year of the assessment relative to the first. We compare these results to those of other states administering the same exam; we also compare the first and second year s of the CST to the se first year s of the Smarter Balanced assessment . Next, we examine district and school results for the cohort of students going from 4th to 5th grade , taking advantage of the fact that Smarter Balanced scores can be compared across grades in a meaningful way . We first compare achievement levels over time, then consider the growth in student test scores , and finally we examine the two measures in combination, using the state’s new academic accountability metric. Controlling for demographic characteristics, we ask which districts and schools are performing better or worse than their peers, and whether they were also outliers on the previous years’ standardized t ests. LCFF targets funding to districts, but because results at individual schools can often vary widely from the district average , we examine the differences in achievement and growth between schools and their districts. More Students Are Meeting Achievement Standards, but Gaps Persist The Smarter Balanced assessments measure how students are doing relative to the state’s new academic standard s— in other words, they measure whether students have t he knowledge and skills needed for success in future coursework in each academic area. While the test results are sorted into four achievement levels— not meeting, nearl y meeting, meeting, and exceeding the standard—our analysis in this section focuses only on 1 The Smarter Balanced test is a part of the California Assessment of Student Performance and Progress (CAASPP), which also inc ludes English Language Arts and Mathematics tests in Spanish and tests for students with disabilities, in addition to tests in science. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 6 meeting or not meeting the standard. I n both English Language Arts and Mathematics, more of California’s students met the state standard for achievement in the second year of Smarter Balanced testing (Figure 1). Among all tested students —those in grades 3–8 and grade 11 —the share meeting the stan dard rose by 5 percentage points in English (from 44% to 49%) and by 2 percentage points in Math (from 35% to 37%). And at each grade level, the percentage of students meeting the standard was higher in 2016 than in 2015. In English, s tudents in grades 3 a nd 4 had lower achievement than the other grades, while students in grade 11 were the highest. In math , students in grades 3 and 4 had higher achievement , while lower shares of students in other grades —particularly grade 5— met the standard. FIGURE 1 In every grade tested , more students met the standard in the second year of Smarter Balanced testing SOURCE: California Department of Education. Although test performance has improved compared to 2014–15, as students and teachers adjust to the new curric ulum and assessments , many students are quite far from meeting the state standard . A closer look at the state’s 5 th grade results provides more detail on how students fared on the second year of tests. We can follow the cohort of 4th graders assessed in the 2014 –15 school year to the 5th grade in the 2015– 16 school year to get a sense of how students have done on the tests over two years. We do not have individual student test results, so we cannot be certain these are exactly the sam e students—some probably move in and out of schools, districts, or even the state from one school year to the next . One reason we focus on this cohort is that most students remain in the same school as they go from 4 th to 5 th grade, rather than graduating to middle or junior high school as they do in the higher grades . 2 In addition, the 4th and 5 th grades are common years for English Learners to be reclassified, so it is important to examine test scores in these years. For a point of comparison, the results from the analysis below are presented for the 7th –8th grade cohort in Technical Appendix A . About half of the state’s 5 th graders met the standard in English, with the share incr easing from 40 percent of 4th grade students in 2014–15 to 49 percent of 5th grade students in 2015–16. The share of students in the 5 th grade 2 In addition, we compared key demographic characteristics of 4 th graders (in 2015) and 5th graders (in 2016) and found that the groups were roughly unchanged at the district level in terms of income and race/ethnicity. 0 20 40 60 All 34567811Share of students who met standard (%) Grade English Language Arts 0 20 40 60 All 34567811Share of students who met standard (%) Grade Mathematics 2014– 15 2015–16 PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 7 who met the standard for math actually fell by 2 percentage points compared to the share of 4th grade rs who met the standard in the year before. Interestingly, the same is true for the other states that use the Smarter Balanced test —shares of 5 th grade students meeting the math standard are lower than the share of 4 th grade students meeting the standard in the prior year. Early Smarter Balanced Test Results in California Are Similar to Results in Other States— and to the Early Years of the CST With the shift to the Smarter Balanced tests, California joined a group of 14 other states using the common statewide assessment in English and math. Another group of six states and the District of Columbia use a different statewide assessment developed by the Partnership for Assessment of Readiness for College and Careers, or PARCC. Both tests are aligned with the Common Core State Standards . Achievement levels for California’s 5 th grade students were slightly lower than those seen in Michigan, Washington, Oregon, and Connecticut, the four other large -population states that use the Smarter Balanced assessment ( Figure 2). The share of students meeting the standard in California was 49 percent in English and 33 percent in math, while the average for the four other large states was about 56 percent and 41 percent respectively . In English, the share of students meeting the standar d rose by 9 percentage points in California compared to an average of 5 percentage points in the other states. All five states saw achievement in math for 5 th grade students fall relative to 4 th grade students in the year before, though California fell by less than the other states . 3 FIGURE 2 California’s results were slightly below the average of other large states SOURCES: California Department of Education ; Connecticut State Department of Education ( 2016); Michigan Department of Education. ( 2016 ); Ore gon State Department of Education (2016); Washington State Office of Superintendent of Public Instruction (2016). 3 The share of students meeting the standard for 5 th graders could be lower because 5th grade math concepts are more difficult, because the cut points are set too high, or for a number of other reasons. An additional point of reference, California’s 2015 NAEP math scores held steady for both 4 th and 8 th graders relative to 2013. 0 10 20 30 40 50 60 70Share of students who met standard (%) English Language Arts 0 10 20 30 40 50 60Share of students who met standard (%) Mathematics 2014-15, 4th grade 2015-16, 5th grade PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 8 The second- year results also show that the state’s teachers and students are adjusting to new standards somewhat more quickly than in the past . California last shift ed to new standards in 1997, and a new assessment, the California Standards Test (CST), was adopted in 1998. In the first years that CST achievement levels were reported for English and math ; about 36 percent of 5th grade students met the standard in English —the same share as 4 th grade students the prior year —while the share of students meeting the standard in math fell from 37 percent to 35 percent for 5th grade students relative to the previous cohort. On the Smarte r Balanced test, more students met the standard in English in the first year and the share increased by more in the second , though the results for math are largely the same as in the early years of the CST . Of course, the curriculum, standards, and assessm ent tools all differ, so these comparisons should be interpreted with caution. High -Need Students Are Improving Slowly We now take a closer look at economically disadvantaged students and English Learners, since improving their educational outcomes is a major focus of the state’s new funding and accountability systems. The shares of high- need students meeting the English and math standards were lower than the share s of all students meeting those standards (Figure 3 ). However, t he 10 percentage point incre ase in the share of economically disadvantaged students meeting the standard in English was slightly higher than the increase among all students , reducing the achievement gap from 36 to 35 percentage points. FIGURE 3 Students saw substantial increases in achievement in English Language Arts, but not Mathematics SOURCE: California Department of Education. Author calculations . NOTE: The English -o nly group consists of students who are fluent only in English —students who speak a language other than English at home are not included. Economically disadvantaged students are typically those who qualify for free or reduced-price school meals. 020 406080100 Current and Former EL Reclassified ELEnglish Only English Learner (EL) Not Economically Disadvantaged Economically Disadvantaged All Students Share of students who met the standard (%) English Language Arts0 20 406080100 Current and Former EL Reclassified ELEnglish Only English Learner (EL) Not Economically Disadvantaged Economically Disadvantaged All Students Share of students who met the standard (%) Mathematics 2014-15, 4th Grade 2015-16, 5th Grade PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 9 English Learner s saw only a slight increase in achievement —the share meeting the standard in English rose from 11 percent to 13 percent. I n math, the share meeting the standard was 8 percent—down from 11 percent in the year before. The achievement gap between English Learner s and English-only students increased by 6 percentage points in English but remai ned the same in math. 4 The low level of achievement for English Learner s can be partly explained by features of the English Learner program itself. English Learner students are reclassified as fluent in English based —in part—on their scores on tests of academic English. Thus, as English Learner students achieve better results, they are removed from the group, keeping the average from rising. 5 As shown in Figure 3, the combined achievement of English Learner and reclassified students is much greater than that of English Learner s alone— their results are similar to those of economically disadvantaged students. 6 Test Score Growth Is Uneven In addition to looking at the shares of students who met achievement standards in the second year of Smarter Balanced tes ts, we can also look at the growth in student scores . Scores on the Smarter Balanced test, which range from about 2,000 to 3,000 points, are reported along a “vertical” scale, which allows comparisons of scores over time for individual students as well as for groups of students . Th e points, known as “scale scores ,” are divided into ranges that align with the four Smarter Balanced achievement level s. Figure 4 shows the cut points for the different achievement levels in English. FIGURE 4 The scale scores co rresponding to a given achievement level rise with each grade SOURCE: California Department of Education . 4 Note that while the achievement gap between English Learner and English -only students is somewhat smaller than the gap for economically disadvantaged and other students on the Mathe matics test, the gaps for the English Language Arts test were somewhat larger. English Learners tend to show higher levels of achievement in Mathematics than in English Language Arts. 5 New federal regulations will adjust the criteria for English Learner status to include both current English Learners and all students who have been reclassified in the prior four years. 6 Of course, there is a large degree of overlap between the two groups —approximately 75 percent of English Learners are economically disadva ntaged. 2000 2200 2400 2600 2800 3000 34567811 Mean scale score Grade Smarter Balanced Achievement Levels (English Language Arts) Standard Exceeded Standard Met Standard Nearly Met Standard Not Met PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 10 Specifically, the cut point for meeting the standard in English for 4 th grade students is 2,473, while the cut point in the 5 th grade is 2,502. Thus, a student just above the cut point for meeting the standard would need to increase her scale score by 29 points to remain at the same achievement level as she moves from the 4 th to the 5 th grade. In math , she would need to increase her score by 43 points to keep pace. On average , scale score growth was about 50 points in English , but only about 31 points in math—not enough to keep pace with the rising math standard. High-N eed Students’ Scale Score Growth Is Relatively Low Whether they are e conomically disadvantaged or not, students had similar levels of score growth in English — about 50 and 53 points, respectively . The gap in growth was somewhat larger in math : scores for students who are economically disad vantaged increased by only 28 points , compared to 40 points for those who are not. At 14 points on English and 19 points on math, the gaps in growth between English Learner and English-only students were large on both tests. Reclassified students also saw below -average growth on both tests, though the combined group of English Learner and reclassified students showed more growth than both groups individually. This is partly because, in 2015–16, 47 percent of this group was reclassified students, compared to just 34 percent in the year before . This underlines the importance of considering reclassified students in assessments of the results for English Learner s. Lower -Achieving Districts and Schools Have the Least Growth The second- year test results show that achievement gaps are not narrowing substantially —in fact, they may even be widening . When we consider the relationship between the share of students meeting the standard (achievement) and the rise in scale scores (growth), we find that low -achieving districts also tend to have low growth. In Figure 5, we plot English results for 5 th grade students across California’s school districts , with achievement on the horizontal axis and scale score growth on the vertical axis . 7 The gray-shaded corners contain the districts in the top or bottom 20 percent in both achievement and growth, highlight ing the districts at the extremes of both measures. The orange lines denote the averages of the two mea sures. About 49 percent of students at the average district met the achievement standard while average scale score growth was about 48 points— well above the 29 points needed to meet the standard moving from the 4th to the 5th grade . 7 This approach uses the method used by the CORE districts for inspiration. See CORE Districts, “Select Information about the C ORE Growth Model,” presentation slides , October 31, 2016. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 11 FIGURE 5 District results in English Language Arts show wide variation in achievement and growth SOURCE: Author calculations using 2016 California Department of Education data . NOTE: Achievement refers to the share of students who met the state standard. Growth refers to the difference in scale scores for 5 th grade students in the 2015 –16 school year and 4 th grade students in the 2014 –15 school year. Sample limited to the 583 districts that serve at least 50 students in the 5 th grade . The coefficient from the reg ression of share meeting the standard on growth is 0.12, meaning that a 10 percentage point increase in achievement is associated with a 1.2 point increase in the average scale score. The p -value and R 2 of the regression are 0.00 and 0.05 respectively. There is a slightly positive relationship between the share of students meeting the standard at a district and the average scale score growth, indicating that high -achievement districts also tend to be high-growth and low - achievement districts are often low -g rowth . This is a troubling sign that the gap between the districts that are doing well —those in the top right corner —and those that are not —the ones in the bottom left corner —could be growing. However, the spread of the districts shows that individual cases vary widely. At the district level , economically disadvantaged students saw below -average achievement but average growth, while English Learner s saw below-average achievement and growth. Figure 6 shows that the relationship between the two measures is stronger for both groups of high -need students. L ower achieving districts often saw lower growth among both their economically disadvantaged students and their English Learner and reclassified students . That relationship is also visible in the extremes . Among all students, about 3.2 percent were at districts in the lower left corner of Figure 5 , indicating that they were both low -achievement and low -growth . But for economically disadvantaged students that number was 3.9 percent , and for English Learners a nd reclassified students combined, the number was 5.9 percent . PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 12 FIGURE 6 High -need students saw below -average results in English Language Arts SOURCE: Author calculations using 2016 California Department of Education data . NOTE S: Achievement refers to the share of students that met the state standard. Growth refers to the difference in scale scores for 5 th grade students in the 2015 –16 school year and 4 th grade students in the 2014 –15 school year. Sample limited to the 583 districts that serve at least 50 students in the 5 th grade. For economically disadvantaged students, the coefficient from the regression of share meeting the standard on growth is 0.39, meaning that a 10 percentage point increase in achievement is associated with a 3.9 point increase in the average scale score. The p -value and R 2 of the regression are 0.00 and 0.15 respectively. For English Learners and reclassified students, the coefficient from the regression of share meeting the standard on growth is 0.08, meaning that a 10 percentage point increase in achievement is associated with a 0.8 point increase in the average scale score. The p -value and R 2 of the regression are 0.01 and 0.02 respectively. When we compare achievement and growth among high-need students in districts that receive LCFF concentration grants and those in districts that do not get this extra funding, we find a similar relationship between the two. This suggests that high -need students in school districts targeted for extra L CFF funding are not falling behind high -need students in districts that have fewer high-need students (see Technical Appendix B ). Compared to districts, there is much more variatio n in school-level results. There are greater shares of school s at every extreme (Table 1) . Just as achievement and growth are very far from the statewide average in some districts , results for some schools are very different from the ir district average. I ndividual student results are probably even more disparate. TABLE 1 S chools saw larger shares of students at performance extremes in English Language Arts Low-achievement, low -growth Low-achievement, high- growth High-achievement, low -growth High-achievem ent, high-growth Districts 3.2% 1.7% 0.9% 2.5% Schools 6.2% 1.9% 2.6% 3.9% SOURCE: Author calculations using 2016 California Department of Education data . About 7.6 percent of economically disadvantaged students and 6.7 percent of English Learners attend low - achievement, low -growth schools. As was the case with districts, the se shares are higher than the 6.2 percent of all students who are at low -achievement, low -growth schools. The share of students in schools and districts at the low end of both achievement and growth is an important indicator of whether the state’s recent educational reforms are meeting their goal of narrowing achiev ement gaps. Of course, there will always be some schools or PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 13 districts at the low end of either achievement or growth. The big concern is that schools or districts at the low end of both measures at the same time typically have high shares of high -need students and are the most in need of improvement , yet are falling further behind their peers. The New State Performance Measure Combines A chievement and Growth To bring California’s accountability system in line with its other K –12 reforms, s tate education agencies have developed a new academic accountability measure that incorporates achievement and growth into a single metric, which we refer to as performance. The new performance measure translates achievement into five status levels and growth into five change levels, placing slightly more emphasis on status . 8 The combination of a given status level and change level corresponds to one of the five colors that make up the performance measure. Moving from the lowest to the highest level, the colors are red, orange, yellow, green , and blue. 9 We used the new performance measure to look at results for 5 th graders, and found that only 4 percent of districts are at red, the lowest performance level , while about 11 percent of schools are at that level. By design, the most common category for both is yellow, with 47 percent of districts and 38 percent of schools at that performance level. About 15 percent of districts and 19 percent of schools are at blue, the top performance level . Districts contain many more stude nts than schools so their averages can mask some of the variation in student performance. The school results , which show more students at either extreme , are more likely to represent the actual distribution of individual student performance . TABLE 2 Districts and schools with high shares of high -need students had low performance levels Districts Schools Share at performance level Average district share of high- need students at performance level Share at performance level Average school share of high- need students at performance level Red (Lowest) 4% 80% 11% 86% Orange 12% 72% 16% 76% Yellow 47% 78% 38% 80% Green 22% 48% 16% 50% Blue (Highest) 15% 23% 19% 33% SOURCE: California Department of Education —author calculations based on proposed academic indicator presented by CDE at January 2017 meeting of the State Board of Education. NOTE : The table is based only on 5th grade results in English Language Arts , not results for entire district s or school s. 8 “Change” is meant to describe differences in mean scale score s among 5th grade students in 2015 –16 and 5 th graders in 2014 –15. In the previous sections, “growth” refers to the differences in scale scores among the approximate cohort of 5 th grade students in 2015 –16 and 4 th grade students in 2014 –15. 9 The state is also developing color -coded indicators for a variety of measures in addition to Smarter Balanced scores, including graduation and suspension rates. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 14 We found that districts and schools with higher shares of high-need students tend to be at lower performance levels. About 80 percent of the students in red districts are economically disadvantaged or English Learner s, while only 23 percent of the students in the blue districts are high need. At red schools , 86 percent of students are high need , compared to only 33 percent of blue -school students . Accounting for Student Demographics Highlights Different Successes and Struggles While the new state performance measure does include growth, it does not account for student demographics. Our analysis has shown that districts and schools with large percentages of high -need students are not performing as well as those with lower percent ages of high-need students, but we have found notable exceptions. These exceptions could serve as useful examples for struggling districts and schools . Some districts have much higher performance levels than their share of high- need students would lead us to expect , while others have not perform ed as well as districts with similar student populations . 10 When we examine the interaction of student demographics and performance levels, we find that a few districts with large high -need populations —such as Rosemead Elementary in Los Angeles and Savanna Elementary in Orange County, where 86 percent and 77 percent of students are high need—are at the blue performance level . T he other districts at that level are , on average, about 23 percent high need. D istricts with small high-need populations —such as Rancho Santa Fe Elementary in San Diego and Spreckels Union Elementary in Monterey , where 5 percent and 15 percent of students are high need—were at the yellow and orange performance levels, where the average share of high-nee d students was above 70 percent . The state measure’s color scheme would mask this district’s relative underperformance since a yellow or orange performance level might not raise much concern despite being such a poor result for a district with thes e demographics. Full tables with the top 20 district outliers in both directions —along with the top 50 school outliers —are in Technical Appendices C and D. D istricts tend to move in and out of the top or bottom 20 each year. However, some districts have been able to maintain a high level of success over time and—in a pair of cases —across the transition from the CST to the Smarter Balanced test. Of the top 20 districts that performed at a high level despite their share of high -need student s on the 2015–16 Smarter Balanced tests , five also overperformed on the 2014– 15 tests. 11 Two of those districts, Rosemead Elementary and Heber Elementary, were also in the top 20 on the CST in the 2012–13 school year. The results for underperforming districts are more complicated . Three districts in the bottom 20 in the second year of the new tests were also in the bottom 20 in the first year, and two of these three (Muroc Joint Unified and Spreckel s Union Elementary ) were in the bottom 20 on the CST in 2013. 12 The results for these districts were only slightly below the statewide average, but they lagged behind other districts with similarly low shares of high -need students. While the top- performing outlier districts saw excellent results even though they have student populations that tend to not do as well on average, the outlier districts at the other end did not have objectively poor results —rather, they only failed to meet the high expectations predicted for a district with their low shares of high -need students. 10 Outliers were determined using predicted values from a multiple regression with categorical variables for e ach performance level on the share of high-need students at a district. The R2 value of the regression was 0.52. Those districts where the share of high -need students most differed from the share that would be predicted based on their performance level are listed as outliers. 11 Rosemead Elementary (Los Angeles), Heber Elementary (Im perial), Lawndale Elementary (Los Angeles), Westminster (Orange), and Little Lake Elementary (Los Angeles) were among the top 20 outliers in both the 2015– 16 and the 2014–15 school years. 12 Spreckels Union Elementary (Monterey), Union Hill Elementary (Neva da), and Muroc Joint Unified (Kern) were among the bottom 20 outliers in both the 2015–16 and the 2014– 15 school years. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 15 Many Schools Are Underperforming their Districts If we apply the state’s performance measure at both the district and the school level , we can see how conditions at individual schools in a district can differ—sometimes greatly —from the district overall. Wide variation in performance across a district’s schools may have implications for how district LCFF funding is spent. Focusing first on low -performing schools, we find that most schools at lower performance levels in English are in districts with different overall performance levels—a bout 74 percent of red schools are in districts that are at the yellow performance level or better (Figure 7). At the higher end of the scale , more blue schools match up with their district- wide performance levels— though they are more often in districts at the green level. Just 2 percent of top- performing schools are in districts performing at the orange level, and none are at the red level. FIGURE 7 Struggling schools tend to underperform their districts SOURCE: California Department of Education —author calculations based on proposed academic indicator presented by CDE at January 2017 meeting of the State Board of Education. NOTE: Performance levels for English Language Arts are based on 5 th grade results only, not entire district s or school s. More than 82 percent of schools are at the same performance level as their district or only one level removed. Of the remaining 18 percent of schools, most are at least two performance levels lower than their districts: almost 12 percent of schools perform far below the district average, while only about 6 percent of schools are far above it. More than 80 percent of the schools that are two or more performance levels below their districts are red schools in yellow districts and orange schools in green districts . The vast majority (73%) of schools that perform far above their districts are blue schools in yellow districts. Schools that are far below their district performance average might learn most from demographically similar schools that exceed expectations. The s hare of high -need students at a district or school explains most of the difference between school and district performance . Schools that perform far above their districts have shares of high -need students that are almost 21 percentage points below the district-wide share on average. In schools that underperform their districts , the shares of high- need students are 11 percentage points higher. The wide range of results —even among schools within a district —highlight s the challenges of using test results to make sure that academic success is broadly shared. 7% 19% 63% 11% Red schools (lowest performance level) 2% 22% 42% 34% Blue schools (highest performance level) District performance level PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 16 Conclusion In the second year of evaluating students’ mastery of California’s new standards, we find evidence of progress that is similar in magnitude to the progress between the first and second year of t he CST. Scores could be improving for many reasons —including greater familiarity with the new online testing and better implementation of the Common Core curriculum. S cores have improved for most students, but achievement gaps persist and have widened in s ome cases. This highlights the challenges that led to the state’s emphasis on providing additional funding and services to the neediest students (economically disadvantaged students, English Learner s, and foster youth) . Districts and schools that are seeing both low growth and low achievement are particularly troubling —those results show that students who were already lagging their peers could be falling further behind. Schools and districts with high share s of economically disadvantaged students and English Learners are more likely to be in this group than other schools and districts . However, our findings suggest that high-need students at high -need districts are not losing ground when compared to high- need students in low-need districts. Another hope ful sign is that the achievement gap did not widen in all cases— in fact, the gap narrowed for some districts and schools. These schools and districts could be a beacon for demographically similar districts and schools where the gap is widest or has grown the most. High-need schools faring well in high -need districts could be particularly helpful. Also, the results for English Learners suggest that the reclassification process will need to be adjusted to take account of the difficulties students have had with the new test. The Local Control and Accountability Plans require districts to continuously improve outcomes , and though there is some guidance on this coming from the state, the infrastructure to assist these districts is still evolving. The state need s to expect and facilitat e improvements not just in high -need districts but also in high -need schools within low -need districts. Standardized tests are a familiar benchmark of student achievement and the Smarter Balanced assessments are expli citly linked to the state’s new academic standards, b ut test results are just one measure of school success. Other measures are either in use or being considered. Educators and policymakers are rightly concerned about improving achievement and increasing g rowth while working to understand how other measures of academic success and social and emotional learning will help student s succeed . PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 17 REFERENCES California Department of Education. 2014‒ 2016. CALPADS Unduplicated Pupil Count (UPC) Source File and Student Poverty FRPM Data . California Department of Education. 2016. CAASPP 2016 Test Results for English Language Arts/Literacy and Mathematics. California Department of Education. 2017. “ Proposed Performance Standards for the Academic Indicator Based on Student Test Scores in English Language Arts/Literacy and Mathematics for Grades Three through Eight, Definition of the English Learner Subgroup (Item 02) .” P resented at California State Board of Education January meeting . Connecticut State Department of Education. 2016. Smarter Balanced Reporting Information . CORE Districts. 2016. Select Information about the CORE Growth Model . Education Week. 2016. “State Testing: An Interactive Breakdown of 2015 -16 Plans.” Hill, Laura and Iwunze Ugo . 2016 . High -Need Students and California’s New Assessments . Public Policy Institute of California. Michigan Department of Education. 2016. Student Assessment . Oregon Department of Education. 2016. Assessment Results. Smarter Balanced Assessment Consortium. 2017. Reporting Scores . US Department of Education. Institute of Education Sciences , National Center for Education Statistics . National Assessment of Educational Progress (NAEP), various years, 2013 and 2015 Mathematics Assessments. Washington S tate Office of Superintendent of Public Instruction. 2016. Washington State Report Card. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 18 ABOUT THE AUTHOR S Iwunze Ugo is a research associate at the Public Policy Institute of California. His work focuses on K –12 education. Previously, he studied adolescent health and changes in education spending following the passage of Proposition 13. He holds a BA in economics and mat hematics, with a minor in statistical science, from the University of California, Santa Barbara. Laura Hill is a senior fellow at the Public Policy Institute of California. Her areas of expertise are K –12 education and immigration. She is currently researching English Learners in California schools, the implementation of the Local Control Funding Formula, and special education . Prior to joining PPIC, she was a research associate at the SPHERE Institute and a National Institute of Aging postdoctoral fellow. She holds a PhD in demography and an MA in e conomics from the University of California, Berkeley. ACKNOWLEDGMENTS We wish to thank Paul Warren, Jim Lanich, Jake Jackson, Edgar Cabral, and Lynette Ubois for their thoughtful reviews and Mary Severance, Kate Reber, and Jenny Miyasaki for editorial support. We also thank Theresa Garcia for many helpful conversations. Any errors are our own. PUBLIC POLICY INSTITUTE OF CALIFORNIA Board of Directors Mas Masumoto, Chair Author and Farmer Mark Baldassare President and CEO Public Policy Institute of California Ruben Barrales President and CEO GROW Elect María Blanco Executive Director Undocumented Student Legal Services Center University of California Office of the President Louise Henry Bryson Chair Emerita, Board of Trustees J. Paul Getty Trust A. Marisa Chun Partner McDermott Will & Emery LLP Chet Hewitt President and CEO Sierra Health Foundation Phil Isenberg Former Chair Delta Stewardship Council Donna Lucas Chief Executive Officer Lucas Public Affairs Steven A. Merksamer Senior Partner Nielsen, Merksamer, Parrinello, Gross & Leoni, LLP Gerald L. Parsky Chairman Aurora Capital Group Kim Polese Chairman ClearStreet, Inc. Gaddi H. Vasquez Senior Vice President, Government Affairs Edison International Southern California Edison The Public Policy Institute of California is dedicated to informing and improving public policy in California through independent, objective, nonparti san research. Public Policy Institute of California 500 Washington Street, Suite 600 San Francisco, CA 94111 T: 415.291.4400 F: 415.291.4401 PPIC.ORG PPIC Sacramento Center Senator Office Building 1121 L Street, Suite 801 Sacramento, CA 95814 T: 916.440.1120 F: 916.440.1121" } ["___content":protected]=> string(104) "

R 0417IUR

" ["_permalink":protected]=> string(106) "https://www.ppic.org/publication/student-achievement-and-growth-on-californias-k-12-assessments/r_0417iur/" ["_next":protected]=> array(0) { } ["_prev":protected]=> array(0) { } ["_css_class":protected]=> NULL ["id"]=> int(9046) ["ID"]=> int(9046) ["post_author"]=> string(1) "1" ["post_content"]=> string(0) "" ["post_date"]=> string(19) "2017-05-20 02:43:21" ["post_excerpt"]=> string(0) "" ["post_parent"]=> int(6603) ["post_status"]=> string(7) "inherit" ["post_title"]=> string(9) "R 0417IUR" ["post_type"]=> string(10) "attachment" ["slug"]=> string(9) "r_0417iur" ["__type":protected]=> NULL ["_wp_attached_file"]=> string(13) "R_0417IUR.pdf" ["wpmf_size"]=> string(6) "469773" ["wpmf_filetype"]=> string(3) "pdf" ["_edit_lock"]=> string(12) "1495273592:1" ["wpmf_order"]=> string(1) "0" ["searchwp_content"]=> string(44910) "APRIL 2017 Iwunze Ugo and Laura Hill Supported with funding from the Evelyn and Walter Haas, Jr. Fund Student Achievement and Growth on California’s K –12 Assessments © 2017 Public Policy Institute of California PPIC is a public charity. It does not take or support positions on any ballot measures or on any local, state, or federal legislation, nor does it endorse, support, or oppose any political parties or candidates for public office. Short sections of text, not to exceed three paragraphs, may be quoted without written permission provided that full attribution is given to the source. Research publications reflect the views of the authors and do not necessarily reflect the views of our funders or of the staff, officers, advisory councils, or board of directors of the Public Policy Institute of California. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 3 Introduction 5 More Students Are Meeting Achievement Standards, but Gaps Persist 5 Test Score Growth Is Uneven 9 The New State Performance Measure Combines Achievement and Growth 13 Conclusion 16 References 17 About the Authors 18 Acknowledgments 18 Technical appendices to this paper are available on the PPIC website. California’s recent emphasis on local control for public K –12 schools recognizes that the work of improving student performance is largely the responsibility of school districts. The Local Control Funding Formula (LCFF) and the Local Control and Accountabi lity Plans (LCAPs) provide the resources and structure to help districts meet their goals, with a particular focus on closing achievement gaps. Yet in the first few years, LCAPs have been criticized for having poorly articulated goals, especially for high -need students. Now that two years of data from California’s new standardized test are available, we are in a better position to evaluate early implementation of both the Common Core State Standards and the new Local Control Funding Formula, particularly its impact on economically disadvantaged students and English Learners. We find that California’s school children did much better in the second year of the new statewide standardized tests, though achievement gaps have not substantially narrowed.  About 49 percent of students met grade -level achievement standards in English Language Arts, while 37 percent met the standard in Mathematics. Compared to the other large states using the same test, those shares were somewhat lower, but California’s incre ases from the year before were nearly twice as large.  Economically disadvantaged students and English Learners also largely showed improvement in the shares of students meeting the standards — although those increases were not at rates fast enough to close substantial achievement gaps with students who are not disadvantaged.  High -need districts (where over 55 percent of students are economically disadvantaged or English Learners) saw lower levels of achievement on average, with about 33 percent of students meeting the standards compared to about 60 percent for districts with fewer high -need students.  While most districts saw similar growth regardless of their share of disadvantaged students, districts with low levels of achievement and growth tended to have higher shares of high-need students. These results are especially troubling because they indicate that disadvantaged students are falling further behind. Another way to understand school and district performance is to compare performance after adjusting f or student demographics. We find that the districts and schools that either exceeded or failed to meet expectations according to estimates based on their student demographics often differed from those identified as outliers by the state’s new academic acco untability measure. We also find CONTENTS SUMMARY PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 4 that some schools have exceeded or failed to meet expectations repeatedly over the past several years. Finally, we find that results at many schools outperformed or —more often —lagged behind the overall results in their dist ricts. This report can help districts assess student performance—overall and for high- need students in particular— compared to other districts with similar demographics. The districts and schools with better -than- expected performance on the Smarter Balance d tests can be a valuable resource to the districts and schools that are still struggling to implement the new state standards and adapt to their new responsibilities to improve their own accountability plans. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 5 Introduction The second year of California’s statewide administration of the Smarter Balanced1 assessment allows parents, educators, and policymakers a second look at achievement and a first look at growth for K–12 students as measure d by this standardized test . Test results are a useful tool for assessing the early implementation of both the new C ommon C ore standards and district Local Control and Accountability Plans (LCAPs) . The good news is that most students are making progress. I n order to address long-standing achievement gaps, districts receive extra funding based on the share of their students who are high -need (defined as economically disadvantaged, English Learner or foster youth) through the new Local Control Funding Formula (LCFF) . For this and other reasons, we are particularly interested in understanding the progress that these high -need students are making. Note that s tandardized assessments are only one way to measure student success, albeit an important one. The State B oard of Education is developing a new accountability system with multiple measures, including academic achievement and growth, as well as social and emotional learning. When we examined the first -year Smarter Balanced results, we found that achievement —the share of students meeting the state’s academic standards—was lower than it was in the final year of the old statewide assessments, the California Standards Test or CST (Hill and Ugo 2016) . This was not unexpected given the new test content and design, as well as the new standards. We also found that high -need students had some of the lowest scores, and the achievement gaps between those students and others who are not disadvantaged were larger than they were on the CST. Using the second year assessment, districts and schools can measure both achievement and growth . In this report , we first describe how California’s students performed on the second year of the assessment relative to the first. We compare these results to those of other states administering the same exam; we also compare the first and second year s of the CST to the se first year s of the Smarter Balanced assessment . Next, we examine district and school results for the cohort of students going from 4th to 5th grade , taking advantage of the fact that Smarter Balanced scores can be compared across grades in a meaningful way . We first compare achievement levels over time, then consider the growth in student test scores , and finally we examine the two measures in combination, using the state’s new academic accountability metric. Controlling for demographic characteristics, we ask which districts and schools are performing better or worse than their peers, and whether they were also outliers on the previous years’ standardized t ests. LCFF targets funding to districts, but because results at individual schools can often vary widely from the district average , we examine the differences in achievement and growth between schools and their districts. More Students Are Meeting Achievement Standards, but Gaps Persist The Smarter Balanced assessments measure how students are doing relative to the state’s new academic standard s— in other words, they measure whether students have t he knowledge and skills needed for success in future coursework in each academic area. While the test results are sorted into four achievement levels— not meeting, nearl y meeting, meeting, and exceeding the standard—our analysis in this section focuses only on 1 The Smarter Balanced test is a part of the California Assessment of Student Performance and Progress (CAASPP), which also inc ludes English Language Arts and Mathematics tests in Spanish and tests for students with disabilities, in addition to tests in science. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 6 meeting or not meeting the standard. I n both English Language Arts and Mathematics, more of California’s students met the state standard for achievement in the second year of Smarter Balanced testing (Figure 1). Among all tested students —those in grades 3–8 and grade 11 —the share meeting the stan dard rose by 5 percentage points in English (from 44% to 49%) and by 2 percentage points in Math (from 35% to 37%). And at each grade level, the percentage of students meeting the standard was higher in 2016 than in 2015. In English, s tudents in grades 3 a nd 4 had lower achievement than the other grades, while students in grade 11 were the highest. In math , students in grades 3 and 4 had higher achievement , while lower shares of students in other grades —particularly grade 5— met the standard. FIGURE 1 In every grade tested , more students met the standard in the second year of Smarter Balanced testing SOURCE: California Department of Education. Although test performance has improved compared to 2014–15, as students and teachers adjust to the new curric ulum and assessments , many students are quite far from meeting the state standard . A closer look at the state’s 5 th grade results provides more detail on how students fared on the second year of tests. We can follow the cohort of 4th graders assessed in the 2014 –15 school year to the 5th grade in the 2015– 16 school year to get a sense of how students have done on the tests over two years. We do not have individual student test results, so we cannot be certain these are exactly the sam e students—some probably move in and out of schools, districts, or even the state from one school year to the next . One reason we focus on this cohort is that most students remain in the same school as they go from 4 th to 5 th grade, rather than graduating to middle or junior high school as they do in the higher grades . 2 In addition, the 4th and 5 th grades are common years for English Learners to be reclassified, so it is important to examine test scores in these years. For a point of comparison, the results from the analysis below are presented for the 7th –8th grade cohort in Technical Appendix A . About half of the state’s 5 th graders met the standard in English, with the share incr easing from 40 percent of 4th grade students in 2014–15 to 49 percent of 5th grade students in 2015–16. The share of students in the 5 th grade 2 In addition, we compared key demographic characteristics of 4 th graders (in 2015) and 5th graders (in 2016) and found that the groups were roughly unchanged at the district level in terms of income and race/ethnicity. 0 20 40 60 All 34567811Share of students who met standard (%) Grade English Language Arts 0 20 40 60 All 34567811Share of students who met standard (%) Grade Mathematics 2014– 15 2015–16 PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 7 who met the standard for math actually fell by 2 percentage points compared to the share of 4th grade rs who met the standard in the year before. Interestingly, the same is true for the other states that use the Smarter Balanced test —shares of 5 th grade students meeting the math standard are lower than the share of 4 th grade students meeting the standard in the prior year. Early Smarter Balanced Test Results in California Are Similar to Results in Other States— and to the Early Years of the CST With the shift to the Smarter Balanced tests, California joined a group of 14 other states using the common statewide assessment in English and math. Another group of six states and the District of Columbia use a different statewide assessment developed by the Partnership for Assessment of Readiness for College and Careers, or PARCC. Both tests are aligned with the Common Core State Standards . Achievement levels for California’s 5 th grade students were slightly lower than those seen in Michigan, Washington, Oregon, and Connecticut, the four other large -population states that use the Smarter Balanced assessment ( Figure 2). The share of students meeting the standard in California was 49 percent in English and 33 percent in math, while the average for the four other large states was about 56 percent and 41 percent respectively . In English, the share of students meeting the standar d rose by 9 percentage points in California compared to an average of 5 percentage points in the other states. All five states saw achievement in math for 5 th grade students fall relative to 4 th grade students in the year before, though California fell by less than the other states . 3 FIGURE 2 California’s results were slightly below the average of other large states SOURCES: California Department of Education ; Connecticut State Department of Education ( 2016); Michigan Department of Education. ( 2016 ); Ore gon State Department of Education (2016); Washington State Office of Superintendent of Public Instruction (2016). 3 The share of students meeting the standard for 5 th graders could be lower because 5th grade math concepts are more difficult, because the cut points are set too high, or for a number of other reasons. An additional point of reference, California’s 2015 NAEP math scores held steady for both 4 th and 8 th graders relative to 2013. 0 10 20 30 40 50 60 70Share of students who met standard (%) English Language Arts 0 10 20 30 40 50 60Share of students who met standard (%) Mathematics 2014-15, 4th grade 2015-16, 5th grade PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 8 The second- year results also show that the state’s teachers and students are adjusting to new standards somewhat more quickly than in the past . California last shift ed to new standards in 1997, and a new assessment, the California Standards Test (CST), was adopted in 1998. In the first years that CST achievement levels were reported for English and math ; about 36 percent of 5th grade students met the standard in English —the same share as 4 th grade students the prior year —while the share of students meeting the standard in math fell from 37 percent to 35 percent for 5th grade students relative to the previous cohort. On the Smarte r Balanced test, more students met the standard in English in the first year and the share increased by more in the second , though the results for math are largely the same as in the early years of the CST . Of course, the curriculum, standards, and assessm ent tools all differ, so these comparisons should be interpreted with caution. High -Need Students Are Improving Slowly We now take a closer look at economically disadvantaged students and English Learners, since improving their educational outcomes is a major focus of the state’s new funding and accountability systems. The shares of high- need students meeting the English and math standards were lower than the share s of all students meeting those standards (Figure 3 ). However, t he 10 percentage point incre ase in the share of economically disadvantaged students meeting the standard in English was slightly higher than the increase among all students , reducing the achievement gap from 36 to 35 percentage points. FIGURE 3 Students saw substantial increases in achievement in English Language Arts, but not Mathematics SOURCE: California Department of Education. Author calculations . NOTE: The English -o nly group consists of students who are fluent only in English —students who speak a language other than English at home are not included. Economically disadvantaged students are typically those who qualify for free or reduced-price school meals. 020 406080100 Current and Former EL Reclassified ELEnglish Only English Learner (EL) Not Economically Disadvantaged Economically Disadvantaged All Students Share of students who met the standard (%) English Language Arts0 20 406080100 Current and Former EL Reclassified ELEnglish Only English Learner (EL) Not Economically Disadvantaged Economically Disadvantaged All Students Share of students who met the standard (%) Mathematics 2014-15, 4th Grade 2015-16, 5th Grade PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 9 English Learner s saw only a slight increase in achievement —the share meeting the standard in English rose from 11 percent to 13 percent. I n math, the share meeting the standard was 8 percent—down from 11 percent in the year before. The achievement gap between English Learner s and English-only students increased by 6 percentage points in English but remai ned the same in math. 4 The low level of achievement for English Learner s can be partly explained by features of the English Learner program itself. English Learner students are reclassified as fluent in English based —in part—on their scores on tests of academic English. Thus, as English Learner students achieve better results, they are removed from the group, keeping the average from rising. 5 As shown in Figure 3, the combined achievement of English Learner and reclassified students is much greater than that of English Learner s alone— their results are similar to those of economically disadvantaged students. 6 Test Score Growth Is Uneven In addition to looking at the shares of students who met achievement standards in the second year of Smarter Balanced tes ts, we can also look at the growth in student scores . Scores on the Smarter Balanced test, which range from about 2,000 to 3,000 points, are reported along a “vertical” scale, which allows comparisons of scores over time for individual students as well as for groups of students . Th e points, known as “scale scores ,” are divided into ranges that align with the four Smarter Balanced achievement level s. Figure 4 shows the cut points for the different achievement levels in English. FIGURE 4 The scale scores co rresponding to a given achievement level rise with each grade SOURCE: California Department of Education . 4 Note that while the achievement gap between English Learner and English -only students is somewhat smaller than the gap for economically disadvantaged and other students on the Mathe matics test, the gaps for the English Language Arts test were somewhat larger. English Learners tend to show higher levels of achievement in Mathematics than in English Language Arts. 5 New federal regulations will adjust the criteria for English Learner status to include both current English Learners and all students who have been reclassified in the prior four years. 6 Of course, there is a large degree of overlap between the two groups —approximately 75 percent of English Learners are economically disadva ntaged. 2000 2200 2400 2600 2800 3000 34567811 Mean scale score Grade Smarter Balanced Achievement Levels (English Language Arts) Standard Exceeded Standard Met Standard Nearly Met Standard Not Met PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 10 Specifically, the cut point for meeting the standard in English for 4 th grade students is 2,473, while the cut point in the 5 th grade is 2,502. Thus, a student just above the cut point for meeting the standard would need to increase her scale score by 29 points to remain at the same achievement level as she moves from the 4 th to the 5 th grade. In math , she would need to increase her score by 43 points to keep pace. On average , scale score growth was about 50 points in English , but only about 31 points in math—not enough to keep pace with the rising math standard. High-N eed Students’ Scale Score Growth Is Relatively Low Whether they are e conomically disadvantaged or not, students had similar levels of score growth in English — about 50 and 53 points, respectively . The gap in growth was somewhat larger in math : scores for students who are economically disad vantaged increased by only 28 points , compared to 40 points for those who are not. At 14 points on English and 19 points on math, the gaps in growth between English Learner and English-only students were large on both tests. Reclassified students also saw below -average growth on both tests, though the combined group of English Learner and reclassified students showed more growth than both groups individually. This is partly because, in 2015–16, 47 percent of this group was reclassified students, compared to just 34 percent in the year before . This underlines the importance of considering reclassified students in assessments of the results for English Learner s. Lower -Achieving Districts and Schools Have the Least Growth The second- year test results show that achievement gaps are not narrowing substantially —in fact, they may even be widening . When we consider the relationship between the share of students meeting the standard (achievement) and the rise in scale scores (growth), we find that low -achieving districts also tend to have low growth. In Figure 5, we plot English results for 5 th grade students across California’s school districts , with achievement on the horizontal axis and scale score growth on the vertical axis . 7 The gray-shaded corners contain the districts in the top or bottom 20 percent in both achievement and growth, highlight ing the districts at the extremes of both measures. The orange lines denote the averages of the two mea sures. About 49 percent of students at the average district met the achievement standard while average scale score growth was about 48 points— well above the 29 points needed to meet the standard moving from the 4th to the 5th grade . 7 This approach uses the method used by the CORE districts for inspiration. See CORE Districts, “Select Information about the C ORE Growth Model,” presentation slides , October 31, 2016. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 11 FIGURE 5 District results in English Language Arts show wide variation in achievement and growth SOURCE: Author calculations using 2016 California Department of Education data . NOTE: Achievement refers to the share of students who met the state standard. Growth refers to the difference in scale scores for 5 th grade students in the 2015 –16 school year and 4 th grade students in the 2014 –15 school year. Sample limited to the 583 districts that serve at least 50 students in the 5 th grade . The coefficient from the reg ression of share meeting the standard on growth is 0.12, meaning that a 10 percentage point increase in achievement is associated with a 1.2 point increase in the average scale score. The p -value and R 2 of the regression are 0.00 and 0.05 respectively. There is a slightly positive relationship between the share of students meeting the standard at a district and the average scale score growth, indicating that high -achievement districts also tend to be high-growth and low - achievement districts are often low -g rowth . This is a troubling sign that the gap between the districts that are doing well —those in the top right corner —and those that are not —the ones in the bottom left corner —could be growing. However, the spread of the districts shows that individual cases vary widely. At the district level , economically disadvantaged students saw below -average achievement but average growth, while English Learner s saw below-average achievement and growth. Figure 6 shows that the relationship between the two measures is stronger for both groups of high -need students. L ower achieving districts often saw lower growth among both their economically disadvantaged students and their English Learner and reclassified students . That relationship is also visible in the extremes . Among all students, about 3.2 percent were at districts in the lower left corner of Figure 5 , indicating that they were both low -achievement and low -growth . But for economically disadvantaged students that number was 3.9 percent , and for English Learners a nd reclassified students combined, the number was 5.9 percent . PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 12 FIGURE 6 High -need students saw below -average results in English Language Arts SOURCE: Author calculations using 2016 California Department of Education data . NOTE S: Achievement refers to the share of students that met the state standard. Growth refers to the difference in scale scores for 5 th grade students in the 2015 –16 school year and 4 th grade students in the 2014 –15 school year. Sample limited to the 583 districts that serve at least 50 students in the 5 th grade. For economically disadvantaged students, the coefficient from the regression of share meeting the standard on growth is 0.39, meaning that a 10 percentage point increase in achievement is associated with a 3.9 point increase in the average scale score. The p -value and R 2 of the regression are 0.00 and 0.15 respectively. For English Learners and reclassified students, the coefficient from the regression of share meeting the standard on growth is 0.08, meaning that a 10 percentage point increase in achievement is associated with a 0.8 point increase in the average scale score. The p -value and R 2 of the regression are 0.01 and 0.02 respectively. When we compare achievement and growth among high-need students in districts that receive LCFF concentration grants and those in districts that do not get this extra funding, we find a similar relationship between the two. This suggests that high -need students in school districts targeted for extra L CFF funding are not falling behind high -need students in districts that have fewer high-need students (see Technical Appendix B ). Compared to districts, there is much more variatio n in school-level results. There are greater shares of school s at every extreme (Table 1) . Just as achievement and growth are very far from the statewide average in some districts , results for some schools are very different from the ir district average. I ndividual student results are probably even more disparate. TABLE 1 S chools saw larger shares of students at performance extremes in English Language Arts Low-achievement, low -growth Low-achievement, high- growth High-achievement, low -growth High-achievem ent, high-growth Districts 3.2% 1.7% 0.9% 2.5% Schools 6.2% 1.9% 2.6% 3.9% SOURCE: Author calculations using 2016 California Department of Education data . About 7.6 percent of economically disadvantaged students and 6.7 percent of English Learners attend low - achievement, low -growth schools. As was the case with districts, the se shares are higher than the 6.2 percent of all students who are at low -achievement, low -growth schools. The share of students in schools and districts at the low end of both achievement and growth is an important indicator of whether the state’s recent educational reforms are meeting their goal of narrowing achiev ement gaps. Of course, there will always be some schools or PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 13 districts at the low end of either achievement or growth. The big concern is that schools or districts at the low end of both measures at the same time typically have high shares of high -need students and are the most in need of improvement , yet are falling further behind their peers. The New State Performance Measure Combines A chievement and Growth To bring California’s accountability system in line with its other K –12 reforms, s tate education agencies have developed a new academic accountability measure that incorporates achievement and growth into a single metric, which we refer to as performance. The new performance measure translates achievement into five status levels and growth into five change levels, placing slightly more emphasis on status . 8 The combination of a given status level and change level corresponds to one of the five colors that make up the performance measure. Moving from the lowest to the highest level, the colors are red, orange, yellow, green , and blue. 9 We used the new performance measure to look at results for 5 th graders, and found that only 4 percent of districts are at red, the lowest performance level , while about 11 percent of schools are at that level. By design, the most common category for both is yellow, with 47 percent of districts and 38 percent of schools at that performance level. About 15 percent of districts and 19 percent of schools are at blue, the top performance level . Districts contain many more stude nts than schools so their averages can mask some of the variation in student performance. The school results , which show more students at either extreme , are more likely to represent the actual distribution of individual student performance . TABLE 2 Districts and schools with high shares of high -need students had low performance levels Districts Schools Share at performance level Average district share of high- need students at performance level Share at performance level Average school share of high- need students at performance level Red (Lowest) 4% 80% 11% 86% Orange 12% 72% 16% 76% Yellow 47% 78% 38% 80% Green 22% 48% 16% 50% Blue (Highest) 15% 23% 19% 33% SOURCE: California Department of Education —author calculations based on proposed academic indicator presented by CDE at January 2017 meeting of the State Board of Education. NOTE : The table is based only on 5th grade results in English Language Arts , not results for entire district s or school s. 8 “Change” is meant to describe differences in mean scale score s among 5th grade students in 2015 –16 and 5 th graders in 2014 –15. In the previous sections, “growth” refers to the differences in scale scores among the approximate cohort of 5 th grade students in 2015 –16 and 4 th grade students in 2014 –15. 9 The state is also developing color -coded indicators for a variety of measures in addition to Smarter Balanced scores, including graduation and suspension rates. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 14 We found that districts and schools with higher shares of high-need students tend to be at lower performance levels. About 80 percent of the students in red districts are economically disadvantaged or English Learner s, while only 23 percent of the students in the blue districts are high need. At red schools , 86 percent of students are high need , compared to only 33 percent of blue -school students . Accounting for Student Demographics Highlights Different Successes and Struggles While the new state performance measure does include growth, it does not account for student demographics. Our analysis has shown that districts and schools with large percentages of high -need students are not performing as well as those with lower percent ages of high-need students, but we have found notable exceptions. These exceptions could serve as useful examples for struggling districts and schools . Some districts have much higher performance levels than their share of high- need students would lead us to expect , while others have not perform ed as well as districts with similar student populations . 10 When we examine the interaction of student demographics and performance levels, we find that a few districts with large high -need populations —such as Rosemead Elementary in Los Angeles and Savanna Elementary in Orange County, where 86 percent and 77 percent of students are high need—are at the blue performance level . T he other districts at that level are , on average, about 23 percent high need. D istricts with small high-need populations —such as Rancho Santa Fe Elementary in San Diego and Spreckels Union Elementary in Monterey , where 5 percent and 15 percent of students are high need—were at the yellow and orange performance levels, where the average share of high-nee d students was above 70 percent . The state measure’s color scheme would mask this district’s relative underperformance since a yellow or orange performance level might not raise much concern despite being such a poor result for a district with thes e demographics. Full tables with the top 20 district outliers in both directions —along with the top 50 school outliers —are in Technical Appendices C and D. D istricts tend to move in and out of the top or bottom 20 each year. However, some districts have been able to maintain a high level of success over time and—in a pair of cases —across the transition from the CST to the Smarter Balanced test. Of the top 20 districts that performed at a high level despite their share of high -need student s on the 2015–16 Smarter Balanced tests , five also overperformed on the 2014– 15 tests. 11 Two of those districts, Rosemead Elementary and Heber Elementary, were also in the top 20 on the CST in the 2012–13 school year. The results for underperforming districts are more complicated . Three districts in the bottom 20 in the second year of the new tests were also in the bottom 20 in the first year, and two of these three (Muroc Joint Unified and Spreckel s Union Elementary ) were in the bottom 20 on the CST in 2013. 12 The results for these districts were only slightly below the statewide average, but they lagged behind other districts with similarly low shares of high -need students. While the top- performing outlier districts saw excellent results even though they have student populations that tend to not do as well on average, the outlier districts at the other end did not have objectively poor results —rather, they only failed to meet the high expectations predicted for a district with their low shares of high -need students. 10 Outliers were determined using predicted values from a multiple regression with categorical variables for e ach performance level on the share of high-need students at a district. The R2 value of the regression was 0.52. Those districts where the share of high -need students most differed from the share that would be predicted based on their performance level are listed as outliers. 11 Rosemead Elementary (Los Angeles), Heber Elementary (Im perial), Lawndale Elementary (Los Angeles), Westminster (Orange), and Little Lake Elementary (Los Angeles) were among the top 20 outliers in both the 2015– 16 and the 2014–15 school years. 12 Spreckels Union Elementary (Monterey), Union Hill Elementary (Neva da), and Muroc Joint Unified (Kern) were among the bottom 20 outliers in both the 2015–16 and the 2014– 15 school years. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 15 Many Schools Are Underperforming their Districts If we apply the state’s performance measure at both the district and the school level , we can see how conditions at individual schools in a district can differ—sometimes greatly —from the district overall. Wide variation in performance across a district’s schools may have implications for how district LCFF funding is spent. Focusing first on low -performing schools, we find that most schools at lower performance levels in English are in districts with different overall performance levels—a bout 74 percent of red schools are in districts that are at the yellow performance level or better (Figure 7). At the higher end of the scale , more blue schools match up with their district- wide performance levels— though they are more often in districts at the green level. Just 2 percent of top- performing schools are in districts performing at the orange level, and none are at the red level. FIGURE 7 Struggling schools tend to underperform their districts SOURCE: California Department of Education —author calculations based on proposed academic indicator presented by CDE at January 2017 meeting of the State Board of Education. NOTE: Performance levels for English Language Arts are based on 5 th grade results only, not entire district s or school s. More than 82 percent of schools are at the same performance level as their district or only one level removed. Of the remaining 18 percent of schools, most are at least two performance levels lower than their districts: almost 12 percent of schools perform far below the district average, while only about 6 percent of schools are far above it. More than 80 percent of the schools that are two or more performance levels below their districts are red schools in yellow districts and orange schools in green districts . The vast majority (73%) of schools that perform far above their districts are blue schools in yellow districts. Schools that are far below their district performance average might learn most from demographically similar schools that exceed expectations. The s hare of high -need students at a district or school explains most of the difference between school and district performance . Schools that perform far above their districts have shares of high -need students that are almost 21 percentage points below the district-wide share on average. In schools that underperform their districts , the shares of high- need students are 11 percentage points higher. The wide range of results —even among schools within a district —highlight s the challenges of using test results to make sure that academic success is broadly shared. 7% 19% 63% 11% Red schools (lowest performance level) 2% 22% 42% 34% Blue schools (highest performance level) District performance level PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 16 Conclusion In the second year of evaluating students’ mastery of California’s new standards, we find evidence of progress that is similar in magnitude to the progress between the first and second year of t he CST. Scores could be improving for many reasons —including greater familiarity with the new online testing and better implementation of the Common Core curriculum. S cores have improved for most students, but achievement gaps persist and have widened in s ome cases. This highlights the challenges that led to the state’s emphasis on providing additional funding and services to the neediest students (economically disadvantaged students, English Learner s, and foster youth) . Districts and schools that are seeing both low growth and low achievement are particularly troubling —those results show that students who were already lagging their peers could be falling further behind. Schools and districts with high share s of economically disadvantaged students and English Learners are more likely to be in this group than other schools and districts . However, our findings suggest that high-need students at high -need districts are not losing ground when compared to high- need students in low-need districts. Another hope ful sign is that the achievement gap did not widen in all cases— in fact, the gap narrowed for some districts and schools. These schools and districts could be a beacon for demographically similar districts and schools where the gap is widest or has grown the most. High-need schools faring well in high -need districts could be particularly helpful. Also, the results for English Learners suggest that the reclassification process will need to be adjusted to take account of the difficulties students have had with the new test. The Local Control and Accountability Plans require districts to continuously improve outcomes , and though there is some guidance on this coming from the state, the infrastructure to assist these districts is still evolving. The state need s to expect and facilitat e improvements not just in high -need districts but also in high -need schools within low -need districts. Standardized tests are a familiar benchmark of student achievement and the Smarter Balanced assessments are expli citly linked to the state’s new academic standards, b ut test results are just one measure of school success. Other measures are either in use or being considered. Educators and policymakers are rightly concerned about improving achievement and increasing g rowth while working to understand how other measures of academic success and social and emotional learning will help student s succeed . PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 17 REFERENCES California Department of Education. 2014‒ 2016. CALPADS Unduplicated Pupil Count (UPC) Source File and Student Poverty FRPM Data . California Department of Education. 2016. CAASPP 2016 Test Results for English Language Arts/Literacy and Mathematics. California Department of Education. 2017. “ Proposed Performance Standards for the Academic Indicator Based on Student Test Scores in English Language Arts/Literacy and Mathematics for Grades Three through Eight, Definition of the English Learner Subgroup (Item 02) .” P resented at California State Board of Education January meeting . Connecticut State Department of Education. 2016. Smarter Balanced Reporting Information . CORE Districts. 2016. Select Information about the CORE Growth Model . Education Week. 2016. “State Testing: An Interactive Breakdown of 2015 -16 Plans.” Hill, Laura and Iwunze Ugo . 2016 . High -Need Students and California’s New Assessments . Public Policy Institute of California. Michigan Department of Education. 2016. Student Assessment . Oregon Department of Education. 2016. Assessment Results. Smarter Balanced Assessment Consortium. 2017. Reporting Scores . US Department of Education. Institute of Education Sciences , National Center for Education Statistics . National Assessment of Educational Progress (NAEP), various years, 2013 and 2015 Mathematics Assessments. Washington S tate Office of Superintendent of Public Instruction. 2016. Washington State Report Card. PPIC.ORG Student Achievement and Growth on California’s K‒ 12 Assessments 18 ABOUT THE AUTHOR S Iwunze Ugo is a research associate at the Public Policy Institute of California. His work focuses on K –12 education. Previously, he studied adolescent health and changes in education spending following the passage of Proposition 13. He holds a BA in economics and mat hematics, with a minor in statistical science, from the University of California, Santa Barbara. Laura Hill is a senior fellow at the Public Policy Institute of California. Her areas of expertise are K –12 education and immigration. She is currently researching English Learners in California schools, the implementation of the Local Control Funding Formula, and special education . Prior to joining PPIC, she was a research associate at the SPHERE Institute and a National Institute of Aging postdoctoral fellow. She holds a PhD in demography and an MA in e conomics from the University of California, Berkeley. ACKNOWLEDGMENTS We wish to thank Paul Warren, Jim Lanich, Jake Jackson, Edgar Cabral, and Lynette Ubois for their thoughtful reviews and Mary Severance, Kate Reber, and Jenny Miyasaki for editorial support. We also thank Theresa Garcia for many helpful conversations. Any errors are our own. PUBLIC POLICY INSTITUTE OF CALIFORNIA Board of Directors Mas Masumoto, Chair Author and Farmer Mark Baldassare President and CEO Public Policy Institute of California Ruben Barrales President and CEO GROW Elect María Blanco Executive Director Undocumented Student Legal Services Center University of California Office of the President Louise Henry Bryson Chair Emerita, Board of Trustees J. Paul Getty Trust A. Marisa Chun Partner McDermott Will & Emery LLP Chet Hewitt President and CEO Sierra Health Foundation Phil Isenberg Former Chair Delta Stewardship Council Donna Lucas Chief Executive Officer Lucas Public Affairs Steven A. Merksamer Senior Partner Nielsen, Merksamer, Parrinello, Gross & Leoni, LLP Gerald L. Parsky Chairman Aurora Capital Group Kim Polese Chairman ClearStreet, Inc. Gaddi H. Vasquez Senior Vice President, Government Affairs Edison International Southern California Edison The Public Policy Institute of California is dedicated to informing and improving public policy in California through independent, objective, nonparti san research. Public Policy Institute of California 500 Washington Street, Suite 600 San Francisco, CA 94111 T: 415.291.4400 F: 415.291.4401 PPIC.ORG PPIC Sacramento Center Senator Office Building 1121 L Street, Suite 801 Sacramento, CA 95814 T: 916.440.1120 F: 916.440.1121" ["post_date_gmt"]=> string(19) "2017-05-20 09:43:21" ["comment_status"]=> string(4) "open" ["ping_status"]=> string(6) "closed" ["post_password"]=> string(0) "" ["post_name"]=> string(9) "r_0417iur" ["to_ping"]=> string(0) "" ["pinged"]=> string(0) "" ["post_modified"]=> string(19) "2017-05-20 02:43:21" ["post_modified_gmt"]=> string(19) "2017-05-20 09:43:21" ["post_content_filtered"]=> string(0) "" ["guid"]=> string(51) "http://148.62.4.17/wp-content/uploads/R_0417IUR.pdf" ["menu_order"]=> int(0) ["post_mime_type"]=> string(15) "application/pdf" ["comment_count"]=> string(1) "0" ["filter"]=> string(3) "raw" ["status"]=> string(7) "inherit" ["attachment_authors"]=> bool(false) }