California’s Changing K-12 Accountability Program
California recently joined a number of other states in adopting the Common Core State Standards, which establish new criteria for what students should learn in school. It also joined a consortium of states to develop new tests based on those standards. The new standards are ambitious, and some teachers are concerned they are not prepared to convey the higher-level skills and concepts they contain. The new tests will allow the state to measure gains in each student’s achievement, creating new options for how the state ranks schools. The change will also prompt the state to reassess the value of state tests in high school and its options for holding secondary schools accountable. More changes to the state’s accountability program are likely when Congress reauthorizes the federal education law, and the way the state addresses these current issues will influence the shape of its future accountability program.
California has taken several steps to revise its K–12 accountability program. Beginning in 2014, the state will switch to the Common Core State Standards (CCSS), a set of high-quality standards in English and mathematics developed by and for the states.1 California also joined the Smarter Balanced Assessment Consortium, a group of states that is developing new tests based on the CCSS.2 The state’s accountability program will also be affected by Senate Bill 1458, which requires adding new indicators of student success to the index by 2016.3
It is likely that there will be additional changes in the future. The state’s accountability measure—the Academic Performance Index (API)—will have to be revised to accommodate the Smarter Balanced assessments. In addition, California may need to redesign its accountability program in line with the changing federal perspective. Because Congress has been unable to reauthorize the federal No Child Left Behind (NCLB) Act, the Obama administration developed a “waiver” program that completely restructures the federal accountability program. The waiver program, for instance, requires state-developed accountability measures that contain a broader array of student success data, including the annual growth in student achievement.4 Although only 34 states (not including California) have received a waiver, many of its provisions may be extended to all states if NCLB is reauthorized—much of the program was part of a 2011 bipartisan bill to reauthorize the federal education act.
Redesigning the state’s accountability program raises a host of significant issues. This report outlines three issues that are currently shaping California’s accountability program.
- Preparing teachers for challenging new standards.
- Measuring individual student gains.
- Reassessing the value of the high school CSTs.
The current state and federal accountability programs
School accountability systems are intended to sharpen the system’s focus on improving student performance. Content standards define what students are expected to learn, and tests measure how well students meet these expectations. Test scores are typically the main driver of school performance measures. Performance targets describe what is expected each year, and schools and districts experience a variety of restrictions and interventions that escalate over time if adequate growth is not achieved.
The new tests will allow the state to measure gains in each student’s achievement, creating new options for how the state ranks schools.
The state’s performance measure—the API—is based almost entirely on California Standards Test (CST) scores.5 There are CSTs in English and mathematics in grades 2–11, and science and history CSTs are administered in selected grades.6 Student scores are reported in five performance levels—Advanced, Proficient, Basic, Below Basic, and Far Below Basic.
Test results are blended to create an API score for each school and district. The API scale ranges from 200 to 1,000. The CSTs are not designed to track individual student growth from one grade to the next, so growth is measured as the difference at each grade level between this year’s students and last year’s. The state target for schools—800—is roughly equivalent to 60 percent of students achieving at the Proficient level and the other 40 percent at the Basic level. Schools with scores below 800 are expected to show sufficient progress toward that goal.7
Testing in California goes above and beyond federal requirements. NCLB requires annual testing only in mathematics and English, and only in grades 3–8. To meet NCLB mandates for high school, California uses scores on the CAHSEE and the results from a special 10th-grade science test. The federal accountability program assesses school performance according to the proportion of students scoring at the Proficient or Advanced levels. By 2014, all students at a school must score at the Proficient or Advanced level on NCLB-mandated tests in order for the school to meet the federal performance requirement.8
Preparing teachers for challenging new standards
The Common Core standards and Smarter Balanced tests will address problems that have become apparent with the current system. One of these problems is the tendency to “teach to the test” at the expense of conceptual understanding and problem-solving ability. Not all educators are prepared to teach the ideas and skills contained in the new standards, however.
As Figure 1 shows, statewide API scores have grown steadily over the past eight years. A synthesis of research found that standards and assessments contribute to this growth by increasing teachers’ focus on tested subjects.9 Consistent with this finding, a study of nine California middle schools concluded that API growth was the result of teachers aligning instruction with the content of state tests, not better teaching methods or students’ deeper understanding of the material.10
Figure 1. Statewide API scores have grown steadily
Aligning instruction with state tests is a common teacher response. Research suggests that multiple-choice tests, such as the CSTs, encourage teachers to focus on alignment.11 As a result, state tests often reduce the curriculum to “a set of things to be known” rather than skills and concepts to be mastered.12 The new standards and tests intend to counter that problem by focusing attention on both knowledge of facts and mastery of concepts.
The greater focus on tested subjects also contributes to a narrowing of the curriculum, reducing time devoted to subjects that are not tested.13 The Common Core standards address this problem by promoting the use of math and language skills in other subjects.14 The English standards, for instance, include requirements for literacy in science and history.15
The CCSS represents a significant departure from the current standards, and teachers who reviewed the new standards expressed concern about the profession’s readiness. “The math teachers in particular were very clear that they had never been taught math in the manner necessary for successfully teaching to the new standards, that they had never taught in that manner themselves, and that they were not sure how to go about it.”16
The long-term impact of the new standards seems promising. The near-term challenge faced by teachers, however, will complicate the transition. Experience shows that helping teachers acquire new classroom practices requires sustained training and support.17 The California Department of Education is developing Internet-based professional development training courses.18 Because of the state’s financial difficulties, however, school districts may have limited resources for classroom support.
Measuring individual student gains
The Smarter Balanced consortium of states, which California has joined, is creating tests that use a “common scale” for the English and mathematics tests in grades 3–8. This design will allow states to measure the annual growth of individual students, and will create new options for measuring school performance.
As noted above, the API uses a cross-cohort growth model that measures the change in scores at each grade level from last year to this. That is, this year’s CST scores for 4th-graders are compared to last year’s 4th-graders. The tests are not designed to measure individual learning gains, and Figure 2 shows what the CST results looks like for a longitudinal cohort of students (data is for the class that graduates in 2013).19 The proportion of students scoring at Proficient or Advanced is quite inconsistent from year to year. The large swings in scores reflect several factors, including test design, the difficulty of the tested material, and student achievement.
Figure 2. Longitudinal cst results are inconsistent
Performance measures based on individual student gains can avoid distortions caused by student demographics that make cross-cohort models less accurate.20 For example, unlike the cross-cohort method, the individual growth model can account for an increase in the proportion of English Learner students from one year’s 4th-grade cohort to the next. This allows for a more accurate picture of student gains.
States are taking advantage of the Obama waiver flexibility to create innovative school performance measures using individual score gains. Colorado, for instance, uses student growth scores to develop a performance target for each student that is based on attaining a proficient level of performance within several years. The Colorado system also allows simple and accurate comparisons of school and district performance. In California, performance comparisons are often invalid because demographic factors influence the API so heavily.21
Growth data from the Smarter Balanced tests will give California new options for measuring achievement and evaluating school performance. The state’s choices for using student-level gains as an indicator of academic performance range from the simple (including a measure of student growth in the API) to the more complex (calculating individual student growth targets).
Reassessing high school CSTs
California faces the major task of aligning its testing system with the new standards and tests. Depending on the purpose of its current tests, the state could consider other ways to hold high schools accountable that do not rely on the current end-of-course CSTs.
California faces the major task of aligning its testing system with the new standards and tests.
The Smarter Balanced consortium will develop English and mathematics assessments in grades 3–8 and 11—per NCLB requirements.22 As discussed above, California also tests English, mathematics, science, and history in grades 9–11, and none of these tests are used in the federal accountability system. Thus, a major issue for the state is whether it should maintain a more robust testing system in the future or reduce the amount of testing that is not required by federal law.23
In evaluating the state’s choices, it is important to consider the different uses of test data. At the local level, test results inform the school improvement process and help parents track their children’s progress. At the state level, the API for high schools is based primarily on CST scores. Other states, though, have found alternate ways to hold high schools accountable for improving achievement:
- Florida’s accountability measure gives schools credit for students who receive passing scores on Advanced Placement or International Baccalaureate tests.
- Massachusetts uses its high school NCLB test in English and mathematics as its high school graduation test.
- Colorado requires all students to take a college preparation test, such as the Preliminary SAT (PSAT), to gauge student readiness for college.
Maintaining the existing system of high school tests would require the development of new standards and assessments that match the approach and depth of the CCSS and Smarter Balanced tests. The state high school exit exam would need a similar revision. To determine whether to make that investment, the state should consider the usefulness of the test data at the local level and explore accountability options available to the state that might accomplish the same goals at a lower cost.
The Common Core standards and Smarter Balanced assessment initiative are creating new challenges and opportunities for the state’s K–12 system. Educators are beginning to face the significant demands of conveying new concepts and skills emphasized in the new standards. But the new tests will measure the growth in student progress more accurately and help reduce the distorting influence of demographics on the API. The new tests have also moved the state to reconsider its high school testing program. More changes are likely, especially if NCLB is reauthorized, and the state’s choices on these issues will help shape its approach to future changes in its accountability program.
- The CCSS have been adopted by 45 states.
- The Smarter Balanced consortium consists of 31 states, including Washington, Oregon, Nevada, North Carolina, Michigan, Pennsylvania, New Jersey, and Connecticut. Other states have formed a second testing consortium, called the Partnership for Assessment of Readiness for College and Careers.
- SB 1458 was signed by Governor Brown in September 2012 (Chapter 578, by Senator Steinberg, Statutes of 2012). It requires the state Board of Education to revise the API so that test scores constitute no more than 60 percent of the value of the index for high schools. The bill does not specify the types of non-test data to be added to the API, but it does authorize the use of several performance measures, including student preparedness for college or jobs after graduation and student success in earning sufficient credits to advance to the next grade.
- “Obama Administration Sets High Bar for Flexibility from No Child Left Behind in Order to Advance Equity and Support Reform,” Office of the White House Press Secretary, September 23, 2011.
- California High School Exit Examination (CAHSEE) scores are also factored into the API at the high school level.
- The state administers tests English in grades 2–11, mathematics in grades 2–7 plus algebra, geometry, and algebra II, a “summative” mathematics test, science in grades 4, 7, and chemistry, physics, biology, earth science in grades 9–11, and history in grades 5, 8, and 9–11.
- State growth requirements are calculated as 5 percent of the difference between 800 and last year’s score.
- This, of course, is unlikely to happen in most schools, and is one motivation for the Obama administration’s waiver program.
- Jane Hannaway and Laura Hamilton, Performance-Based Accountability Policies: Implications for School and Classroom Practices (Urban Institute and RAND Corporation, October 2008).
- Heinrich Mintrop and Tina Trujillo, “The Practical Relevance of Accountability Systems for School Improvement: A Descriptive Analysis of California Schools,” Educational Evaluation and Policy Analysis 29 (4): 319–52.
- Hannaway and Hamilton, “Performance-Based Accountability Policies.”
- Mark Wilson, “Assessment for Learning and for Accountability” (Center for K–12 Assessment & Performance Management, Educational Testing Service, 2010), p. 6.
- Hannaway and Hamilton, “Performance-Based Accountability Policies”; Farkas Duffett Research Group, Learning Less: Public School Teachers Describe a Narrowing Curriculum (report for Common Core, March 2012).
- Center for the Future of Teaching & Learning at WestEd, “Willing But Not Yet Ready: A Glimpse of California Teachers’ Preparedness for the Common Core State Standards” (2012).
- Common Core State Standards Initiative, English Language Arts Standards (2012).
- Center for the Future of Teaching and Learning at WestEd, “Willing But Not Yet Ready,” p. 2.
- See, for example, “Successful Long Term Professional Development,” Connected Mathematics Project, Michigan State University.
- The California Commission on Teacher Credentialing is also revising its Teaching Performance Expectations, which describe the skills, knowledge and abilities of candidates for a teaching credential. These expectations form the basis of the Teaching Performance Assessment that new teachers must pass to earn a credential.
- The longitudinal cohort reports the CST results for 2nd graders in 2003, 3rd-graders in 2004, 4th-graders in 2005, and so on. As a result, the scores reflect the same students from year to year with the exception of students who leave or enter the public school system or students who drop out of school.
- Edward Haertel, “Using a Longitudinal Student Tracking System to Improve the Design for Public School Accountability in California” (Stanford University, August 2005).
- Richard Lee Colvin, Measures That Matter: Why California Should Scrap the Academic Performance Index (Education Sector, 2012). See also S. Eric Larsen, Stephen Lipscomb, and Karina Jaquet, Improving School Accountability in California (Public Policy Institute of California, April 2011).
- No Child Left Behind also requires a high school science test, which California administers as a general science test in grade 10.
- This issue is likely to be a focus of the coming legislative session. To assist its deliberations, the legislature directed the state superintendent of public instruction to submit recommendations for the next generation of state tests by November 1, 2012. CDE convened an advisory group and held a series of meetings around the state to solicit input from educators and community members. At the time this report was written, however, the superintendent had not released his recommendations.