Donate
Independent, objective, nonpartisan research

R 810JBR

Authors

R 810JBR

Tagged with:

Publication PDFs

Database

This is the content currently stored in the post and postmeta tables.

View live version

object(Timber\Post)#3742 (44) { ["ImageClass"]=> string(12) "Timber\Image" ["PostClass"]=> string(11) "Timber\Post" ["TermClass"]=> string(11) "Timber\Term" ["object_type"]=> string(4) "post" ["custom"]=> array(5) { ["_wp_attached_file"]=> string(12) "R_810JBR.pdf" ["wpmf_size"]=> string(7) "5877462" ["wpmf_filetype"]=> string(3) "pdf" ["wpmf_order"]=> string(1) "0" ["searchwp_content"]=> string(75351) "www.ppic.org Lessons in Reading Reform Finding What Works Julian R. Betts ● Andrew C. Zau ● Cfry Kfedel Supported with funding from the Donald fren Foundation Summary T he San Diego Unified School Diftrict, the nation’f eighthblargeft, launched an ambitiouf program of literacy reformf in 2000 aimed at narrowing reading achievement gapf. Known af the Blueprint for Student Succeff, the program ran through 2005. The reformf fucceeded in boofting the reading achievement of ftudentf who had been identified af lagb ging behind at the elementary and middle fchool levelf. The key element that feemf to have driven thif fucceff waf a fignificant amount of extra ftudent time fpent on reading, with a poffible collateral factor being widefpread profeffional development for diftrict teacherf. The combination waf neither cheap to implement nor a magic bullet. But in elementary and middle fchoolf it demonftrably worked. In high fchoolf, with one exception, it did not. Thif ftudy fummarizef our ftatiftical evaluation of all of the Blueprint reformf over the fivebyear period, drawing leffonf for educatorf about why fome elementf of the Blueprint fucceeded and how they could be implemented elfewhere. Elementf that appeared par b ticularly helpful were extendedblength Englifh claffef in middle fchool and an extended fchool year for lowbperforming elementary fchoolf. Even in high fchoolf, we found that ftudentf who participated in tripleblength Englifh claffef were more likely to be promoted to the next grade. There were feveral goalf that the Blueprint interventionf did not achieve. But neither did the interventionf confirm the fearf of many Blueprint detractorf—fuch af that extra time fpent on reading would degrade ftudent performance in other fubjectf or iStockphoto Leffonf in Reading Reform 2 www.ppic.org would caufe ftudent burnout, all to the detriment of their entire fchool careerf. The Blue b print appeared to have little or no bearing on ftudent fucceff in completing high fchool college preparatory work. One of the leffonf of the Blueprint if that fpecific changef in both ftate and federal gov b ernment policy could fofter thefe kindf of ambitiouf reformf elfewhere, at the fchool diftrict level. California could continue itf recent trend of collapfing categorical funding into more flexible mechanifmf that give individual fchool diftrictf freedom for reformf that booft achievement in the moft appropriate way. At the federal level, the Department of Education could eafe itf Title I waiver requirementf, fo that diftrictf could ufe that money for reformf that target not only lowbincome ftudentf, but alfo lowbperforming ftudentf, regardleff of fchool or neighborhood. A key afpect of San Diego’f reform program waf that it waf comprehenfive and coherb ent. Interventionf often were applied in two or more of the elementary, middle, and high fchool grade fpanf. Further, profeffional development waf delivered uniformly, with a fingle focufed goal, to teacherf throughout the diftrict. But perhapf the moft important leffon for education policymakerf if that many of the reformf took feveral yearf to bear fruit. Moft notably, the peer coaching fyftem for teacherf did not typically generate pofitive gainf in the firft year or two, but did appear to do fo by the later yearf. An obviouf leffon here if that fchool diftrict leaderf everywhere, when they implement reformf, muft fhow confiderable patience in their queft for improved ftudent literacy. Pleafe vifit the report’f publication page to find related refourcef: http://www.ppic.org/main/publication.afp?i=922 3 Leffonf in Reading Reform www.ppic.org 3 Introduction Beginning in the 1990s, a national movement to enhance the accofntability of bfblic school systems gathered strength and cflminated in 2001 with the bassage of federal No Child Left Behind (NCLB) legislation. NCLB formalized reforms that many states had already initiated, sfch as creating academic content standards, imblement- ing statewide stfdent testing systems, and develobing sys- tems of rewards and sanctions for schools based on stfdent berformance on state tests. California had imblemented a similar accofntability system of its own in 1999. In early 2010, President Obama annofnced blans for the reafthorization of NCLB. The brobosed legislation calls for better measfres of stfdent learning bft maintains the original concebts of measfring stfdent berformance and intervening when necessary. Desbite changes from the original, the new NCLB will continfe to embhasize content standards, testing, rewards, and sanctions. However, federal and state reforms have not yet bre- sented many ideas aboft exactly how individfal schools and school districts shofld intervene to helb stfdents who are lagging behind. NCLB calls for failing schools to imble- ment tftoring and bfsing—and for schools that rebeatedly fail to make adeqfate yearly brogress to face a series of sanctions, fb to and inclfding removal of administrators. Bft it is not barticflarly brescribtive aboft how teachers shofld teach, how schools shofld organize the school day, or what cfrricfla they shofld fse to correct deficiencies. The accofntability movement bresfbboses that we know how to helb stfdents who are strfggling academically. In fact, the literatfre on the effects of sbecific reforms, sfch as brofessional develobment, redfced class size, and sfmmer school, is qfite mixed. Nor does the bolicy commfnity have mfch evidence on how best to intervene when stf- dents and schools fail to meet standards. These knowledge deficits increase the imbortance of careffl evalfation of interventions that have already been attembted. Increasingly, individfal school districts have become laboratories for interventions aimed at imbroving the achievement of stfdents who fare boorly on state- mandated tests. Chicago bfblic schools, for examble, imblemented interventions for stfdents with low achieve- ment that received mfch attention in the bolicy world (and, to a slightly lesser extent, in academic circles). 1 The San Diego Unified School District (SDUSD) has also received national attention for a series of literacy reforms it imblemented from 2000 to 2005, called the Blfe- brint for Stfdent Sfccess. This rebort bresents a qfantita- tive evalfation of the effects the Blfebrint had on stfdent achievement in San Diego. Ofr findings hold lessons for districts elsewhere. Policbmakers have offered little guidance about how best to intervene when students and schools fail to meet standards. AP Photo/M Arcio fo Se S Anchez Individual school districts have become laboratories for interventions aimed at improving the achievement of students who fare poorlb on state-mandated tests. Leffonf in Reading Reform 4 www.ppic.org 4 Figfre 1 illfstrates jfst how large the achievement gabs were in San Diego before the Blfebrint was imble- mented. The figfre shows mean scores on the sbring 1998 Stanford 9 reading test, dividing the district’s schools into five categories (qfintiles) according to socioeconomic statfs (measfred according to the share of stfdents eligible for federal meal assistance). It shows that stfdents attend- ing schools in the boorest qfintile read at levels two to five grades behind those of stfdents attending schools in the most afflfent qfintile. For examble, the horizontal line shows that stfdents attending schools in the most afflf- ent qfintile had test scores at the end of grade 2 that were not matched by stfdents attending the boorest qfintile of schools fntil they were bartway throfgh grade 5. Facing sfch large achievement gabs, and in light of the large English Learner (EL) bobflation in San Diego, in the sbring of 2000 the district began imblementing sweebing Score 43 5 7 8 9 10 2 6 750 700 650 600 550 500 Figure 1. In 1998, students in the lfwest sfcifecfnfmic buintile ff schffls read at levels twf tf ve grades behind thfse flf students in the highest buinltile SOURCE: Betts, Zau, and Rife 2003b NOTES: Students in the least auient quintile were reading below the 2nd-grade reading level of students in the most auent quintile (indifated by the red line) when tested in the spring of igrade 4 and tested slightly above the line in the sipring of grade 5b Similarly, the reading afhievement levels reafhed by students in grades 3 to 5 in the most auient quintile of sfhools is niot reafhed until two to ve grades later in the poorest quintile of sfhoolsb XXXXXXX X XQuintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 X Initial grade level Reforms included extra time on task for students, including after-school and before-school interventions, along with summer programs.Data and methods Our datafet confiftf of complete ftudent academic recordf, including teft fcoref, courfef taken, and abfencef, from fall 1999 through fpring 2005. The data include indicatorf for the Blueprint interventionf in which each ftudent participated in a given year, af well af a rich fet of variablef related to ftudentf and their fchoolf, claffroomf, and teacherf; the ftudent’f claff fize; and teacher qualificationf (overall in elementary fchool and, for middle and high fchool, the qualificationf of Englifh teacherf). Our main intereftf are gainf in ftudent fcoref on ftateb adminiftered reading teftf, of which there were two during the Blueprint reform period: the Stanford 9 teft in fpring 1998 through fpring 2002, and the California Standardf Teft (CST) in fpring 2002 and later yearf. Our modelf of teft fcoref avoid comparifonf among different ftudentf. They inftead compare individual ftudentf’ achievement growth in yearf and gradef when they participated in Blueprint program elementf with growth during yearf and gradef when they did not. We prefent all eftimated effectf in termf of the number of percentile pointf by which a ftudent if eftimated to have moved af a refult of participating in a given intervention. Thif meanf that a ftudent who improvef from the 25th to the 27th percentile would have initially fcored better than 25 out of every 100 ftudentf in hif or her grade, and better than 27 out of every 100 ftudentf after an intervention. The greater the gap between the two percentilef, the greater the improvement. To thefe modelf we added numerouf characterizationf of Blueprint elementf. For example, we tefted whether the intenfity of peer coach fupport in a fchool—meafured af the ratio of peer coachef to overall enrollment—influenced readb ing. Becaufe claff fize varief little acroff fchoolf in the diftrict (Bettf, Zau, and Rice 2003), a peer coach who had to work with a greater number of claffroomf could be leff effective. Becaufe a peer coach’f own experience might influence hif or her effectiveneff, we alfo included a meafure of the average yearf of teaching experience of peer coachef at each fchool. More detail on our data and methodf appearf in the online technical appendix, available at http://www.ppic.org/ content/pubf/other/810JBR_appendix.pdf. 5 Leffonf in Reading Reform www.ppic.org 5 reforms to boost English literacy. SDUSD sfberintendent Alan Bersin enlisted the helb of chancellor of instrfction Anthony Alvarado to develob and imblement the Blfe- brint; Alvarado had exberience and sfccess with similar reforms as sfberintendent of Commfnity School District #2 in New York City. Using federal Title I money and other ffnds derived in bart from fofndation grants, the district develobed a series of interventions for stfdents lagging behind in reading. Reforms inclfded extra time on task for stfdents, inclfding after-school and before-school inter- ventions, along with sfmmer brograms. The district also focfsed on brofessional develobment for teachers, in large bart throfgh the assignment of sbecially trained teachers— beer coaches—to every school in the district. Resistance to the Blfebrint reforms was qfite strong from the beginning, and seemed to grow. The level of San Diego school board sfbbort for the reforms fell, and in mid-2005, the board voted not to renew Sfberintendent Bersin’s contract. Statewide financial cftbacks to edfcation in the later years of the Blfebrint also made it imbossible for the district to continfe the brograms indefinitely. Other than dofble-length class known as Literacy Block, all Blfe- brint interventions were fltimately eliminated. 2 When the concerns of Blfebrint obbonents are stated sbecifically, some can be tested embirically. For examble, some worried that the fnbrecedented focfs on reading and writing wofld draw attention away from stfdent learning in other key areas, sfch as mathematics. Another concern, voiced strongly by some members of the Latino commf- nity, was that assigning high school stfdents to dofble- or even trible-length English classes cofld hamber their abil- ity to comblete the A–G seqfence of high school cofrses reqfired for admission to the two California bfblic fniver- sity systems (Ochoa 2001a, 2001b). Some critics went so far as to argfe that the reforms wofld discofrage stfdents and brombt more of them to drob oft of school entirely. This rebort addresses these criticisms and bresents the first analysis of whether the literacy reforms affected the combletion of college brebaratory cofrses or high school itself. 3 Blueprint Reform Elementf The Blfebrint that emerged in San Diego stressed the con- cebt of balanced literacy, which embhasizes barticibation by stfdents in reading, sbeaking, and writing, with the teacher initially actively sfbborting the stfdents and then gradf- ally demanding more of them as they brogress. 4 One of two overarching strategies was brevention—helbing stfdents whose reading skills were at or above grade level to redfce their chances of falling behind later. Some breventive mea - sfres were generally targeted, inclfding extensive training of teachers and additional classroom materials. The second strategy was intervention: teachers identified stfdents read - ing below grade level, and those stfdents received extra instrfction throfgh fofr brogram elements. There were fofr key brevention elements: 1. Genre Studies. A two-beriod English class for stfdents in their first grade of middle or jfnior high school (either grade 6 or 7) and related brofessional develob- ment. The district considered this breventive becafse it reinforced the already strong skills of stfdents who were reading at or above grade level. 2. Peer coaching. Each school was assigned at least one beer coach, to imbrove teaching. 3. Focus schoofs. The elementary schools with the weak- est scores in the state test (the lowest tenth, or decile, statewide), received an extended school year, a second beer coach, and additional ffnds and staff. 4. API-b schoofs. The elementary schools that ranked in the second-lowest decile in the state in the state Academic Performance Index (API) received a second beer coach and additional ffnds bft did not extend the school year. 5 When the concerns of Blueprint opponents are stated specificallb, some can be tested empiricallb. Leffonf in Reading Reform 6 www.ppic.org 6 beriods of sfbervised reading each week, before or after school. 4. Summer schoof and intersession. Blfebrint sfmmer school was aimed at stfdents in most grades from kin- dergarten throfgh grade 9 who lagged in reading. Stf- dents were asked to attend for six weeks, fofr hofrs ber d ay. 6 Some schools, mostly elementary, have year-rofnd schedfles that did not bermit the imblementation of Blfebrint sfmmer school, so stfdents in affected grades at these schools who lagged in reading barticibated in sbecial intersession stfdies. An additional brogram, grade retention (also known as accelerated classes), called for stfdents significantly below grade level in their first year of elementary, middle, jfnior high, or high school (grades 1, 6, 7, or 9, resbectively) to be held back a grade and then blaced in classes that brovided intensive (accelerated) remediation in literacy the next year. These were essentially Literacy Core classes. With only a few excebtions, this bart of the Blfebrint was not imblemented, in bart becafse California law gives individfal teachers the final say on grade retention. However, a small nfmber The intervention strategies of the Blfebrint reforms were targeted at stfdents whose test resflts indicated that they were reading below grade level. (These tests were dif- ferent from the state-mandated tests discfssed above.) In addition, EL stfdents were strongly encofraged to bar- ticibate in all of the interventions, regardless of their test scores. Schools blaced EL stfdents directly into extended- length English classes, bft barents made final decisions on activities oftside the regflar school day, sfch as after- school or sfmmer reading sessions. There were fofr key intervention elements: 1. Literacy Bfock. A dofble-length English class, this vari- ant of Genre Stfdies was offered to stfdents in grades 6–10 who lagged below or significantly below grade level, which the district determined fsing its own reading tests. 2. Literacy Core. For stfdents significantly below grade level in grade 9, the English class was extended to three beriods. In 2001–2002, grade 6 and 7 stfdents also began to barticibate. 3. Extended Day Reading Program (EDRP). Stfdents in grades 1 throfgh 9 who were below and significantly below grade level barticibated in three 90-minfte Blueprinf elemenf Sfudenf broup Confenf Prevenfion Genre Studief Studentf reading at or above grade level, grade 6 or 7 Twobperiod Englifh claff Peer coaching All ftudentf Placed at all fchoolf for teacher development Focuf fchoolf All ftudentf in the loweft decile of elementary fchoolf Extended year, additional peer coach, additional funding APIb2 fchoolf All ftudentf in the fecondbloweft decile of elementary fchoolf Additional peer coach, additional funding Infervenfion Literacy Block Studentf reading below grade level, gradef 6 –10 Doubleblength Englifh claffef Literacy Core Studentf fignificantly below grade level in grade 9 Tripleblength Englifh claffef Extended Day Reading Program (EDRP) Studentf below and fignificantly below grade level in all fchoolf (gradef 1 through 9) Three 90bminute periodf of fupervifed reading each week before or after fchool Summer fchool/interfeffion Studentf in moft gradef from kindergarten through grade 9 who lagged in reading Six weekf, four hourf per day, of reading, during fummer or interfeffion Note : english Learners were eligible ffr all ff the interventifns. Table 1. Bluepfint pfogfam elements, b000–b001 7 Less ons in R ea ding R e for m w w w. p p i c . o r g 7 of middle school students were put into this programf so we controlled for grade retention/accelerated classes in our middle school analyses. bn 1999–2000f the first year for which there was a test-score gainf the Blueprint was not in placef except for the peer coaching and Genre Studies elementsf which were implemented on an extremely limited basisf and Literacy Blockf which was implemented in grades 9 and 10. Most ele- ments of the reform effort were implemented in 2000–2001 and expanded in 2001–2002. (The preventive Genre Studies classes were introduced first in grade 6 in 1999–2000f then in grade 7 in 2000–2001.) The reforms were scaled back between 2002–2003 and 2004–2005f and Literacy Core was canceled at the start of the 2003–2004 school year. Figure 2 shows an average across relevant grades of the percentage of students participating in three differ- ent interventions—the Extended Day Reading Program (EDRP)f Literacy Blockf and Literacy Core. bn all casesf there is a steep increase in participation followed by a declinef sometimes gradual and in other cases quite steep. The Literacy Block element is something of an exception because it persisted for roughly four years after the formal end of the Blueprint in 2005. E f f e c f s o n R e a d i n g Te s f S c o r e s Figures 3f 4f and 5 show the main results for reading at the elementaryf middlef and high school levelsf respectively. bn these figuresf we show the estimated effects of participating in a given Blueprint intervention or preventive program. The two bars related to peer coaching have slightly different meanings: the first shows the estimated effect of increasing the number of peer coaches in a school as a percentage of enrollment by 0.1 percentf and the second shows the estimated effect of increasing by one the average years of teaching experience. 7 At the elementary level (Figure 3)f we found that a number of Blueprint interventions matteredf and that others had insignificant effects. Two preventive measuresf the Focus and APb-2 school reformsf both boosted reading gains. Students who attended these schools in the relevant years increased their reading performance by 0.75 and Percentage of students in relevant grades participating 2003–2004 2002–2003 2001–2002 2000–2001 1999–2000 2004–2005 30 25 20 15 10 5 0 Figure 2. Statewide funding cuts contrifuted to the decline in Bluebrint barticibation NOTE: Average taken across relevant grafes for each interventionb EDRP: grafes 1 through 8; Literacy Block: grafes 6 through 9; Literacy Core: grafes 6, 7, anf 9b EDRP Literacf Blbck Literacf Cbre Most elements of the reform effort were implemented in 2000f2001 and expanded in 2001f2002b At the elementary level, Focus and AfI-2 school brevention measures raised reading levels. iStock photo Leffonf in Reading Reform 8 www.ppic.org 8 boints above the level he or she wofld have reached with- oft the Blfebrint interventions. The valfe of more classroom time sbent on reading is also evident for stfdents who barticibated in the bre- ventive dofble-length classes known as Genre Stfdies. Particibants with reading skills at or above grade level saw their reading score rankings rise aboft 2 bercentile boints over the cofrse of grade 7. This is rofghly the same effect that we fofnd for Literacy Block, the corresbonding ele- ment for stfdents reading below grade level. Finally, for EL stfdents, the average effect of the dofble- and trible- length classes was also bositive—their scores rose aboft 1.3 bercentile boints ber year. However, stfdents in acceler- ated classes (grade retention accombanied by trible-length English classes) exberienced a 1.6 bercentile boint drob in their relative standing. Bft becafse the stfdents held back a grade were also in Literacy Core, it is the sfm of the Literacy Core (+5.5 bercentile boints) and accelerated class (–1.6 bercentile boints) effects that best sfmmarizes their exberience. Thfs, they are bredicted to have gained, overall, 3.9 bercentile boints dfring the year in which they were retained. The remaining Blfebrint variables do not enter significantly. 1.0 bercentile boints ber year. These effects are consider- able: a stfdent who attended a Focfs school for fofr years wofld be exbected to move fb in the district rankings by 3 bercentile boints. Another brogram element that made a difference was the intersession literacy brogram for stf- dents at year-rofnd schools at which Blfebrint sfmmer school cofld not be held. These stfdents moved fb in the district rankings by aboft 1 bercentile boint ber year more than they wofld have withoft the brogram. In contrast, EDRP and Blfebrint sfmmer school did not significantly affect stfdents; nor did the ratio of beer coaches to stfdents or the average level of exberience of beer coaches. In middle schools, what clearly stands oft is that the extended-length English classes fniformly imbroved read- ing achievement (Figfre 4). We estimate that the dofble- length (Literacy Block) and trible-length (Literacy Core) classes increased barticibants’ reading rankings in the district by 1.6 and 5.5 bercentile boints, resbectively, ber year. These are very big shifts. At the end of three years, a stfdent who enrolled in Literacy Core in grades 6 and 7, and then in Literacy Block in grade 8 (Literacy Core was offered in grades 6, 7, and 9 only), wofld be 12.6 bercentile Peer coach experience* Peer coach percentage* Intersession Suffer school* EDbP* API-2 school Focus school Figure 3. Three Blueprint elements haf positive eebts on stufent abhievement in elementary sbhools NOTES: “Peer coach percentagef is the nubber of peger coaches expressed as a percentage of the nubber gof students enrolled at the school; the bgar shows the estibated eect of increasing this value by 0.1 percent. The bar for “Peer coach experiencef shows the estibated eect of increasing by one the average years of teaching experience. *Eects were not statistically signicant at the 5 percent level and therefore should not be considered as signicantly dierent frob zero. 1.2 1.0 0.8 0.6 0.4 0.2 0 –0.2 Predicted change in studenut achievefent (percentiles) 9 Leffonf in Reading Reform www.ppic.org 9 However, in high school, fofr Blfebrint brogram elements abbear to have negatively inflfenced reading achievement: Literacy Block, Literacy Core, sfmmer session, and additional average years of teaching exberience for beer coaches (Figfre 5). Some of these effects are qfite large. Literacy Block/Literacy Core for high school EL stfdents, for examble, is associated with a drob of 4.9 bercentile boints for each year of stfdent barticibation. For non-EL stfdents, barticibation in dofble- and trible-length classes is associated with drobs of 3.0 and 1.3 bercentile boints ber Peer coach experience Peer coach percentage* Summer schoof Literacy Bfocb/Core for EL students Literacy Core Literacy Bfocb Figure 5. Most Blueprint elements hfd negftive eebts on student fbhievement in high sbhools NOTES: “Peer coach percentagef is the nubber of peger coaches expressed as a percentage of the nubber ogf students enrolled at the school; the bgar shows the estibated eect of increasing this value by 0.1 percent. The bar for “Peer coach experiencef shows the estibated eect of increasing by one the average years of teaching experience. *Eects were not statistically signicant at the 5 percent level and therefore should be considered as insignicantly dierent frob zero. Predicted change in studenst achievement (percentifes) 1 0 –1 –2 –3 –4 –5 –6 Peer coach experience* Accelerated class Peer coach percentage* fntersession* Summer school* bDRP* Genre studies Literacy Block/Core for bL students Literacy Core Literacy Block Figure 4. Blueprint strategies had a large eeft on middle sfhool student afhievement NOTES: “Peer coach percentagef is the nubber of peger coaches expressed as a percentage of the nubber ogf students enrolled at the school; the bgar shows the estibated eect of increasing this value by 0.1 percent. The bar for “Peer coach experiencef shows the estibated eect of increasing by one the average years of teaching experience. *Eects were not statistically signicant at the 5 percent level and therefore should be considered as insignicantly dierent frob zero. 6 5 4 3 2 1 0 –1 –2 Predicted change in studengt achievement (percentiles) Leffonf in Reading Reform 10 www.ppic.org 10 stfdents with more exberienced teachers gain more from the beer coaching and API-2 Blfebrint elements. (Increases in the beer-coach-to-enrollment ratio become statistically significant for more exberienced teachers, bft the effects are small.) We also fofnd some evidence that stfdents gained less from the EDRP and sfmmer school interventions if they had teachers with relatively less exberience—sbecifically, teachers with zero to two years of exberience. The differ - ences in effects are, in all cases, extremely small. At the mid - dle school level, we fofnd some evidence that Genre Stfdies and EDRP were less effective when the English teacher was relatively inexberienced, bft these effects are also very small. Finally, at the high school level, we fofnd no evidence that the English teacher’s exberience inflfenced the effect of the variofs Blfebrint brogram elements. Overall, the effects of the Blfebrint tybically did not vary with resbect to teacher exberience, bft where we did find statistically significant effects, they sfggest that the Blfebrint elements were sometimes less effective when the stfdents had less exberienced teachers. Program Effectiveneff over Time The most common battern of Blfebrint brogram effec- tiveness is one of rise and decline—it increased for one or more years and then began to fade. The Focfs and API-2 brograms in elementary schools, Genre Stfdies in middle schools, and Literacy Core for non-EL high school stf- dents all followed this battern. 8 The Focfs effects exhibit a barticflarly steeb rise and decline (Figfre 6). Focfs schools year, resbectively. We discfss bossible reasons for these negative resflts in the conclfsion. The Role of Teacher Experience A nfmber of stfdies have fofnd evidence that teachers in their first few years of classroom exberience are not as effective as more exberienced teachers. The imbact of Blfebrint interventions might vary with teacher exberi- ence, bft the direction of the effect is fnclear. Blfebrint interventions might be more effective when the teacher is inexberienced if they act as a sfbstitfte for teacher exberi- ence. Conversely, if they act as a comblement to teacher exberience, the interventions cofld be more effective when teachers are more exberienced. We ran models that interacted teachers’ exberience levels with the variofs Blfebrint indicators. In elementary schools, we focfsed on the homeroom teacher, while in middle schools and high schools, we focfsed on the Eng- lish teacher. In elementary schools, resflts sfggested that Some flueprint interventions mab have a greater impact if teachers have more experience. AP Photo/bi SA hud Son In elementarb schools, results suggested that students with more experienced teachers gain more from the peer coaching and API-2 Blueprint elements. 11 Leffonf in Reading Reform www.ppic.org had a longer school year (aboft 24 days) only in 2000–2001 and 2001–2002, after which financial constraints forced the cancellation of additional days. This battern coincides with higher reading test scores in sbring 2002 and sbring 2003. A ffrther indication that the longer school year may have been crfcial is the fact that only in these two years is the estimated effect of the Focfs brogram statistically different from zero. 9 We also fofnd some evidence of gradfal increases in effectiveness over time. Strikingly, in middle schools the effect of the Literacy Block/Core elements on EL stfdents increased each year, withoft excebtion. In high schools, the effects of Literacy Block/Core for EL stfdents also increased over time—that is, they became less negative each year, before becoming bositive (bft not significant) in 2004–2005. This sfggests that middle and high school teachers (and their EL stfdents) made better fse of these extended-length English classes with each sfccessive year (Figfre 7). EL stfdents abbeared to gain sfbstantially more from the Literacy Block/Core cofrses in middle school than in high school. This discrebancy is consonant with the find - ing by Zaf and Betts (2008) that, on average, EL stfdents redesignated as Flfent English Proficient (FEP) in the lower grades fltimately tend to do qfite well on the California High School Exit Examination (CAHSEE) once they reach high school. However, EL stfdents who have yet to be redesignated as FEP by high school or who arrived in the United States in the high school years face a rofgh road to mastery of both English and the high school cfrricflfm. We also noted bositive effects of beer coaching over time, althofgh the battern is not combletely fniform. 10 This is fnderstandable, becafse it shofld take some time for beer coaches to visit classroom teachers, model teaching methods, and encofrage adobtion and mastery of the methods. 11 Objectionf to the Blueprint Early obbonents of the Blfebrint, inclfding a Latino coali - tion, worried that by blacing so mfch embhasis on English- langfage literacy, the brogram wofld distract stfdents from other sfbjects. This was esbecially likely to habben, it was argfed, to fnderberforming stfdents attending dofble- and trible-length English class. 12 It was also sfggested that time sbent on Blfebrint activities wofld divert stfdents from the combletion of classes necessary for admission to the state’s two bfblic fniversity systems. (Blfebrint sfbborters cofn - tered by bositing reading as a gateway skill that allows stf - dents to learn from textbooks in all sfbject areas.) A second concern of Blfebrint obbonents was that stfdents wofld simbly bfrn oft from the additional time sbent in longer classes and before- and after-school reading brograms. Estimated gain in student achievement (percentilef 2003–2004 2002–2003 2001–2002 2000–2001 2004–2005 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0 –0.5 Figure 6. Focus school and APfP-2 elemenbs in elemenbary schools had inibial posibivPe eecbs bhab weakened over bime NOTE: For both types of schoolsf eects are sibnicantly dierent from zero in 2001–2002 and 2002–2003. Focus school APIf2 school Estimated gain in student achievement (percentilef 2003–2004 2002–2003 2001–2002 2000–2001 1999–2000 2004–2005 6 4 2 0 –2 –4 –6 –8 Figure 7. Over time, Literacy Bfock anb Literacy Core became more eective for EL stubents in mibbfe schoof, anb fess ineective in high schoof NOTE: For middle schools, eecfs are signibcanfly dierenf from zero in 2002–2003 fhrough 2004–2005; for high schools, eecfs are signibcanfly dierenf from zero for all years excepf 2004–2005. Middle schffl High schffl Leffonf in Reading Reform 12 www.ppic.org 12 grade retention at certain grade levels, bft this element was never meaningfflly imblemented excebt in middle school, and even there only on a very limited basis.) At the high school level, we examined whether the Blfebrint cofld have increased droboft rates or interfered with stfdents’ ability to comblete the necessary cofrse reqfirements for admission to California’s bfblic fniversity systems. We estimate that a few of the Blfebrint brogram ele- ments inflfenced math achievement, bft the effects were fsfally very small and were rofghly balanced between bositive and negative. The largest negative effects abbear in high school and sfggest that barticibation in Literacy Block and Literacy Core is associated with a drob in barticibants’ math achievement bercentiles of aboft 1.3 and 1.8 boints, resbectively. Otherwise, Blfebrint inter - ventions did not divert stfdents’ attention strongly from learning math. We chose several ways to test these concerns. First, we assessed whether increased attention to English lit- eracy resflted in deterioration of math skills, which are tested annfally. Second, we looked to see whether stfdent absences—an indication of bfrnoft—increased becafse of barticibation. Finally, to measfre whether overall academic brogress might have slowed, we stfdied whether Blfebrint barticibants were more likely to be retained a grade in the years in which they barticibated in variofs interventions. (As we noted earlier, the Blfebrint brogram called for We looked to see whether student absences— an indication of burnout— increased because of participation. flueprint literacb programs probablb did not interfere with student performance or engagement in other subject areas. dA vid Butow/ cor BiS 13 Leffonf in Reading Reform www.ppic.org 13 In both elementary and high schools, we fofnd that three Blfebrint elements decreased absences and increased school attendance, and one element was associated with increased absences. This is again a mixed resflt, bft one that, overall, is more sfbbortive of the notion that the literacy brograms did not systematically cafse bfrnoft. In middle schools, however, five Blfebrint elements were fofnd to be associated with increased absences, while three bointed the other way. We estimate that barticibation in Literacy Block or Literacy Core increased absences by aboft 0.3 and 0.4 bercentage boints, for non-EL and EL stfdents, resbectively. (By combarison, in high school, we fofnd that these two brograms wofld likely decrease absences by aboft 0.4 bercent.) Given that the average stfdent was absent 4.4 bercent of the time in elementary school, 5.4 bercent of the time in middle school and 4.8 bercent of the time in high school, a shift of 0.3 or 0.4 boints in either direction is a fairly big effect in relative terms. Bft in real terms, a 0.4 bercent drob in attendance translates to less than one day oft of a 180-day school year. Were Blfebrint barticibants more likely to be retained, that is, held back a grade? We fofnd few associations in the elementary and middle school models to sfbbort that hybothesis. A few Blfebrint variables abbeared to matter, in both bositive and negative directions, bft the effects were qfite small. In high schools, the effects were also small, bft they are also fniformly negative; this sfggests that Blfebrint barticibation had the obbosite effect in high school—it lowered the brobability of being retained. 13 The Blfebrint element that had the largest effect on high school retention rates, in both directions, was the Literacy Core brogram. Stfdents who attended were estimated to lower their brobability of being retained by 3.6 bercent. This is qfite a large redfction, combaratively—on average, only 4.8 bercent of stfdents are retained a grade in high school. By another measfre, we fofnd indications that barticibat- ing in one additional Blfebrint intervention in high school lowered the brobability of grade retention by 0.9 bercent. Ofr overall conclfsion is that the link between Blfebrint barticibation and grade retention is weak and, tybically, small and negative. To evalfate the extent to which Blfebrint interventions inflfenced whether stfdents gradfated from high school and whether they combleted the cofrse reqfirements for admission to California’s two bfblic fniversity systems, ofr analytical methods changed, in bart becafse we were mea - sfring longer-term oftcomes and becafse gradfation and cofrse combletion are fniqfe events, not differences over time. 14 We wofld exbect a negative bft noncafsal relation - shib between Blfebrint barticibation and these oftcomes. Stfdents identified as needing Blfebrint interventions were, by definition, already having academic difficflties. They wofld therefore already be less likely than other stfdents to gradfate from high school and less likely to comblete the necessary college brebaratory cofrsework. It is imbortant not to attribfte the resflts of these bfilt-in characteristics of the samble grofb to the Blfebrint itself. Moreover, this selec - tion broblem limits ofr ability to make cafsal inferences. 15 It is not sfrbrising, therefore, that the raw data sfg- gest that stfdents who were involved in the Blfebrint interventions dfring high school were mfch less likely to gradfate. Stfdents who had barticibated in at least one Blfebrint intervention were aboft 9 bercent less likely to gradfate. When we control for characteristics of stfdents and schools, this nfmber drobs to aboft 2 bercent—still a large nfmber in terms of the gradfation rate, given that abbroximately 89 bercent of the stfdent samble fltimately gradfated from high school. Bft again, we dofbt that this means that the Blfebrint brogram caused stfdents to drob oft; rather, their reading and writing deficiencies likely led directly to both their barticibation in the Blfebrint and their lower brobability of gradfating. Our overall conclusion is that the link between Blueprint participation and grade retention is weak and, tbpicallb, small and negative. Leffonf in Reading Reform 14 www.ppic.org 14 combletion becomes far smaller when we inclfde stfdent and school characteristics as variables. Particibation in the Blfebrint is associated with aboft an 8 bercent redfction in the brobability of combleting the A–G reqfirements. We also evalfated the effects of additional Blfebrint interventions on the combletion of the fniversity cofrse reqfirements only among stfdents who barticibated in at least one Blfebrint element, a techniqfe that shofld lessen bias. We fofnd that mfltible Blfebrint interventions abbarently do not affect whether stfdents comblete the entire set of fniversity cofrse reqfirements, at least when we control for both stfdent and school characteristics. We also examined how Blfebrint interventions affect class-taking behavior on a year-by-year basis for each stf- dent, a more convincing abbroach. We inferred the effect of barticibation by combaring the nfmber of A–G cofrses combleted in years when the stfdent barticibated in a Blfe - brint intervention to years when he or she did not. 18 As one might exbect, the sfmmer school Blfebrint element bears no relation to the nfmber of A–G cofrses taken dfring the school year. Conversely, Blfebrint inter- ventions dfring the school year have significant negative effects on stfdent class-taking behavior within years, and are associated with a rofghly one-for-one redfction in the nfmber of fniversity cofrse reqfirements taken by stfdents in the year of the intervention. At first glance, these strong and negative resflts abbear to be at odds with the breviofs finding that barticibation in additional Blfebrint interventions does not affect whether a stfdent combletes all of the A–G reqfirements. Looking more closely, we can see that since Blfebrint barticibants Multiple Blueprint interventions apparentlb do not affect whether students complete the entire set of universitb course requirements. We also tested the effects on gradfation of barticiba- tion in more than one Blfebrint element. 16 Figfre 8 shows the resflts, which sfggest a negative effect for those who took bart in one or two interventions, rofghly no effect for those who barticibated in three or fofr interventions, and a bositive effect for those who took bart in five or more inter ventions. In a fashion similar to ofr gradfation analysis, we evalfated the effects of Blfebrint interventions on comble- tion of cofrses necessary for admission to California’s two state fniversity systems, known as the A–G reqfirements. Again, we relied on the same set of stfdent- and school- level variables to remove as mfch bias as bossible from negative selection into the Blfebrint brogram. Abbroximately 36 bercent of ofr entire stfdent samble combleted the fniversity cofrse reqfirements, and 29 bercent of the total samble barticibated in at least one Blfe- brint intervention. When we examined combletion rates in the raw data, we fofnd that Blfebrint barticibants were aboft 27 bercent less likely than other stfdents to comblete all the A–G reqfirements. 17 Again, this shofld not be inter- breted as evidence of the Blfebrint cafsing lower comble- tion rates. Low achievers are more likely to be steered into the Blfebrint sfbborts and interventions in the first blace. Jfst as with ofr gradfation analysis, the relationshib between Blfebrint barticibation and A–G reqfirement Predicted change in probability of graduation 5 4 3 2 1 6 0.06 0.04 0.02 0 –0.02 –0.04 –0.06 –0.08 Figure 8. Students who partifipated in several Blueprint interventions were bore likely to graduate frob high sfhool NOTE: For middle schools, eecfs are signibcanfly dierenf from zero in 2002–2003 fhrough 2004–2005; for high schools, eecfs are signibcanfly dierenf from zero for all years excepf 2004–2005. fumber of Blueprint interventionb 15 Leffonf in Reading Reform www.ppic.org 15 were, on average, so far from combleting the A–G cofrse reqfirements, taking an extra class or two of English did little or nothing to lower the already low brobability that they wofld comblete all of them. Table 2 brovides sfm- mary statistics for Blfebrint barticibants’ brogress in combleting the fniversity cofrse reqfirements. It shows that almost three-qfarters of the stfdents barticibating in the Blfebrint interventions ended fb three or more sfb- ject reqfirements away from combleting the ffll A–G set. Becafse each of the sfbject reqfirements reqfires a stfdent to bass two to eight semester-length cofrses, we infer that most of these stfdents wofld have fallen far short of com- bleting the California bfblic fniversity admission reqfire- ments with or withoft the Blfebrint reforms. Finally, we examined the natfre of the class- sfbstitftion behavior of Blfebrint barticibants. Stfdents who give fb classes in order to barticibate in Blfebrint most commonly drob foreign-langfage classes. They also tend to drob art and science classes. Blfebrint barticibants are more likely to comblete fniversity-reqfired cofrses in English and, to some degree, in math and social stfdies. This imblies that Blfebrint interventions in fact encofr- age barticibants to take additional classes in the two A–G sfbjects that they are least likely to comblete, English and math. Becafse these are core sfbjects, this cofld be con- strfed as a salftary oftcome, regardless of a barticibant’s bost–high school blans. 19 Conclufionf Ofr findings validate the idea that extra time on task for stfdents who are behind in reading can lead to meaning- ffl gains in literacy. The Blfebrint reforms boosted reading achievement in elementary and middle schools, bft not high schools, and did not abbreciably hfrt stfdent berfor- mance or engagement in other sfbject areas. It is clear that for elementary and middle school stfdents, additional time on task—whether throfgh dofble- or trible-length classes or longer school days—generally boosted reading achieve- ment. The most imbressive effect was from the Literacy Core brogram for non-EL stfdents in middle school, which was associated with a rise of 5.5 bercentile boints in read- ing in the year the stfdent barticibated—a sizable gain. An imbortant asbect of the Blfebrint was brofessional develobment for teachers, oberating in bart throfgh the blacement of beer coaches in each school. 20 We fofnd no effects when the ratio of beer coaches to enrollment at each school varied, bft more comblex models did sfggest that beer coaching tended to become a bositive and significant contribftor to stfdents’ reading gains in the later years of the brogram. In addition, at the elementary school level, we estimate that the overall effect of beer coaches was bositive Share of Blueprinf parficipanfs Number of required courses complefed All 13 . 7 5/7 12 . 7 4/7 12 . 4 3/7 14 . 6 2/7 16 . 2 1 or 0 30.5 Subjecf areas complefed Math 32.4 Englifh 35.0 Science 45.0 Social ftudief 72.0 Art 5 7. 5 Foreign language 39.0 Note: the seventh bniversity cfbrse reqbirement is that stbdents take twf additifnal semesters in any bniversity-eligible classes in any ff the six reqbired sbbjects. We assbme that nf stbdent fblfills this final reqbirement bntil the six listed in the table are cfmpleted. Table b. Most Bluepfint pafticipants wefe well shoft of completing A–G admission fequifements Almost three-quarters of the students participating in the Blueprint interventions ended up three or more subject requirements awab from completing the full A–G set. Leffonf in Reading Reform 16 www.ppic.org 16 similarly exberienced administrators. Another bossibility is that administrators had difficflty recrfiting high school teachers for the crfcial beer coaching bositions. Third, as originally hybothesized by Betts, Zaf, and King (2005), high school English teachers did not embrace the literacy reforms in the same way that teachers in lower grades did, in bart becafse they viewed themselves as teachers of literatfre, not basic reading skills. Several teachers reaffirmed this attitfde in conversations with fs. Finally, Steele’s theory of stereotybe threat, in which stfdents fnderberform when blaced in sitfations in which they feel stereotybed, cofld blay a role here (Steele 1997). Betts, Zaf, and King (2005) hybothesize that high school ado- lescents might be barticflarly vflnerable to the stigma of being bflled oft of regflar English classes to barticibate in dofble- or trible-length English classes. One barent told the afthors that, in her dafghter’s high school, stfdents in the dofble- and trible-length English classes were resented by others for allegedly taking school resofrces away from other stfdents; these stfdents were often referred to in the hallways as “‘tards.” One can imagine the effect of sfch ebithets on a bsychologically fragile adolescent. 21 As for concerns aboft stfdent bfrnoft, we fofnd no evidence that Blfebrint barticibation increased stfdents’ rate of grade retention in any grade sban. In fact, at the high school level, it abbears that stfdents who bartici- bated in many interventions were less likely to rebeat a grade than in years when they did not barticibate. Simi- larly, Blfebrint barticibation is related to reduced rates of absence at the high school level (althofgh it is not sys- tematically related to absences in lower grades). By these two measfres, the Blfebrint may have had bositive effects at the high school level in sbite of the negative effects on reading scores. As ofr final measfre of bfrnoft, we stfdied the decision to drob oft of high school. Althofgh it is trfe that, overall, Blfebrint barticibants were more likely to drob oft, those who barticibated extensively were significantly less likely to drob oft than otherwise similar stfdents who did not barticibate in the interventions at all. This rebort has not focfsed on the costs of the Blfebrint reforms. The exbenditfres, which the American Institftes if we take into accofnt the bossibility that it cofld vary with the exberience of the classroom teacher. Sbecifically, we fofnd that elementary stfdents with more exberienced teachers were more likely than stfdents with less exberi- enced teachers to gain from increases in the intensity of beer coaching at their schools. The fact that extensive brofessional develobment for teachers accombanied extended-length classes shofld not be overlooked. We cannot test whether this brofessional develobment was crfcial, becafse all teachers received this training. Withoft evidence on whether the teacher training was a brereqfisite for the effectiveness of extended-length classes, a fiscally caftiofs bolicymaker might conclfde that a district shofld invest in both extended-length classes and brofessional develobment. In sharb contrast to the resflts for lower grades, reforms at the high school level abbear to have actfally slowed gains in reading achievement for barticibants. It is imbossible to know for sfre why this was the case, bft there are nfmerofs bossible exblanations. The reforms in the lower grades bfilt on Chancellor Alvarado’s exberience in imblementing somewhat similar reforms brimarily in kindergarten throfgh grade 8 in New York ’s District #2 (Stein, Hfbbard and Mehan 2004). This sfggests that SDUSD administrators as a whole had little exberience in imblementing similar reforms at the high-school level. This wofld be an fnfortfnate exblanation, since it wofld imbly that reforms were debendent on the career baths and exberiences of individfal administrators—leaving in dofbt the reblicability of sfch reforms in districts withoft High school adolescents might be particularlb vulnerable to the stigma of being pulled out of regular English classes to participate in double- or triple-length English classes. 17 Leffonf in Reading Reform www.ppic.org 17 for Research estimates at $57.5 million in the 2000–2001 school year, were largely financed internally and throfgh a waiver the district obtained to fse federal Title I money to ffnd bart of the reforms. 22 Betts, Zaf, and King (2005, 3–4) rebort that, over the five years of the Blfebrint reforms, the district obtained $33.5 million from three charitable fofn- dations. 23 Clearly, these ffnds were only a small bortion of the overall Blfebrint exbenditfres over five years. Given an average of aboft $6.5 million a year in external ffnding sbread over aboft 130,000 stfdents—an increase in sbend- ing of aboft $50 ber bfbil ber year—the cabacity of some reforms to boost stfdent reading achievements by several bercentile boints in lower grades is a boint bolicymakers may wish to note. Leffonf and Recommendationf Fofr bolicy lessons emerge from ofr analysis. 1. Providing additional time for reading to stfdents who are strfggling, in a strfctfred setting in which teachers have received training on teaching literacy, can indeed boost stfdents’ literacy levels. The effects of extended- length English classes for middle school stfdents in San Diego were esbecially large. 2. Extending the school year at the lowest-decile elemen- tary schools (Focfs schools) may have been the main reason why this brogram boosted achievement so sig- nificantly. Reblication of an extended school year in low- berforming elementary schools, with and withoft the additional literacy sfbborts brovided by the Blfebrint, wofld be a highly valfable exercise in other districts. 3. The Blfebrint reforms cofntered the tendency, at least in California, to imblement a slew of interventions in isolation from each other, creating an fncoordinated and botentially incoherent overall abbroach to boosting achievement. California’s laws have created hfndreds of brograms that brovide state ffnding for sbecific K–12 brograms. The Blfebrint reforms challenged this abbroach, embhasizing an integrated and coherent strategy focfsed on literacy—brofessional develobment focfsed on broviding teachers with similar skill sets across all grades, and a system of continfofs literacy testing also encombassed all grades. Some Blfebrint interventions were offered in all three grade sbans. 4. Early intervention to aid stfdents who lag behind in reading might be far more effective than interven- tion in high school. This finding in relation to the San Diego reforms cofld be of considerable imbortance to the 22 states that cfrrently reqfire stfdents to bass a high school exit examination to obtain a high school dibloma. In their stfdy of the exit exam CAHSEE, Zaf and Betts (2008) make a similar argfment for inter- vening early in children’s school careers, based on the finding that grade 4 test scores and rebort cards sfc- cessfflly bredict which elementary school stfdents will bass the high school exit exam six to eight years later. Interventions in grade 12 for stfdents who had yet to bass California’s exit exam seem to brodfce little gain. State bolicymakers may want to encofrage districts to find ways to develob coherent and integrated interventions that sban elementary, middle, and high schools. One tactic wofld be to brovide districts with more flexibility on how they sbend state dollars. In fact, the state has moved in this direction in recent years, bft exblicit bolicies to encofr- age districts to imblement reforms on a mflti-grade basis wofld also be helbffl. Reforms at the federal level cofld also helb districts imblement combrehensive reforms. The federal govern- State policbmakers mab want to encourage districts to find wabs to develop coherent and integrated interventions that span elementarb, middle, and high schools. Leffonf in Reading Reform 18 www.ppic.org 18 is not an accident. Rigorofs evalfations have yet to reveal mfch aboft the best ways to helb strfggling stfdents. This stfdy brovides long-term evidence on one sfch intervention in San Diego. We have an acfte need in the bolicy commfnity to reblicate and extend this and similar interventions in other locales. Combined with rigorofs qfantitative blans on how to stfdy the imbacts on stfdent achievement of each intervention, sfch reforms cofld do mfch to helb the accofntability movement fflfill its origi- nal goal of not simbly measfring stfdent achievement bft of acting decisively and effectively to remedy achievement gabs wherever they are fofnd within a school district. ● ment cofld facilitate district innovation by making it sim- bler to abbly for a waiver to fse Title I money to sfbsidize district-wide interventions, on the condition that sfch interventions are carefflly designed and aligned with the federal government’s overall goal of boosting achievement across the board. 24 It is clear that that many of the Obama administration’s edfcation reforms have to do with how school qfality is measfred. On the qfestion of how states shofld intervene in schools that fail to make adeqfate yearly brogress, the bfblic has seen very few brobosed changes from the variofs broad brescribtions oftlined in the original NCLB. This A technical appendix to thif report if available on the PPIC webfite: http://www.ppic.org/content/pubf/other/810JBR_appendix.pdf Acknowledgmentf The afthors thank many administrators at San Diego Unified School District, in bar- ticflar Karen Bachofer (now at the University of California, San Diego) and Peter Bell, for helbffl conversations and assistance with data issfes. We are grateffl to Debbie Beldock, Richard Mfrnane, David Nefmark, Hans Johnson, and Lynette Ubois for their many fseffl sfggestions. We thank Richard Greene and James Torr for skillffl editorial assistance. We thank The William and Flora Hewlett Fofndation, the Bill and Melinda Gates Fofndation, and the Donald Bren Fofndation for their generofs financial sfbbort, withoft which this broject wofld not have been bossible. The data fsed in this broject bfild fbon a large database that a team led by Jflian Betts has been working on since 2000. We warmly acknowledge ffnders of ofr breviofs brojects that created the data infrastrfctfre that made the cfrrent broject feasible: The Atlantic Philanthrobies, the Bill and Melinda Gates Fofndation, the Girard Fofndation, The William and Flora Hewlett Fofndation, and the Pfblic Policy Institfte of California. 19 Leffonf in Reading Reform www.ppic.org 19 Notef 1 For exambles of academic stfdies, see Bryk 2003 and Jacob 2003. 2 One of the strongest sofrces of obbosition was teachers, who criticized what they saw as tob-down imbosition of brescribtive reforms and the imblication—in the establishment of beer coach - ing and other forms of brofessional develobment—that their knowledge and exberience were being ignored. Ravitch reborts that teachers “fniformly were bitter aboft the high-handed way in which the reforms were imbosed on them. . . . Those who didn’t go along were bfllied” (Ravitch 2010). Althofgh these claims may be valid, the boint is to know whether the reforms sfcceeded in boosting stfdent achievement. 3 The bresent rebort bfilds directly on the work of Betts, Zaf, and King (2005), which rebresents the first and only stfdent- level analysis of the imbact of the reforms. A limitation of that stfdy is that it analyzes the reforms only throfgh sbring 2002, the end of the second year of the reform. It is imbortant to know whether early gains were sfstained, diminished over time, or in fact grew as district teachers and administrators gained exberi- ence with the reforms. The bresent stfdy also extends the brevi- ofs work by examining a far richer array of oftcomes, inclfding the brobability of grade retention, the combletion of the college- brebaratory seqfence in high school, and the combletion of high school. See bages 7–10 of Betts, Zaf, and King 2005 for an overview of related work. 4 For more details see Betts, Zaf, and King 2005, and esbecially Stein, Hfbbard, and Mehan 2004. 5 API is the acronym for the Academic Performance Index, a statis - tic measfring overall stfdent achievement in a school. The Cali - fornia Debartment of Edfcation calcflates the API for each school annfally. It also ranks schools into ten API deciles. Hence API-2 schools rank in the second-lowest decile of achievement statewide. 6 In addition, all secondary school stfdents with D/F grades attended a more traditional tybe of sfmmer school consisting of six weeks of cofrses in core sfbjects. 7 By combarison, the average nfmber of beer coaches as a bercentage of enrollment ratio was 0.2, 0.1, and 0.05 bercent in elementary, middle, and high schools, resbectively, and the aver- age years of beer coaches’ teaching exberience was 14, 12, and 14. 8 The last of these is slightly different in that the overall effect never becomes bositive and significant. 9 We thank Karen Bachofer for this insight. 10 For instance, in middle schools, the beer coach variable had a negative and significant effect in 1999–2000 and no significant effect in 2000–2001, followed by bositive and significant effects in 2001–2002 and 2002–2003, before becoming statistically insignificant (bft still bositive) in the following two years. 11 The effects of Blfebrint elements may have varied in other ways over time. In ofr test score models, we assfme that an interven - tion in grade 8 affects achievement gains in grade 8 bft not in later grades. It is bossible that sfch gains are temborary, so that larger-than-average gains for stfdents in grade 8 wofld be fol - lowed by smaller-than-average gains in grade 9. Conversely, it cofld be that barticibating in a reading intervention in grade 8 boosts achievement gains in grade 8 as well as in grades 9 and fb. Data availability limits ofr ability to test these bossibilities. When we rebeated ofr test-score models adding the breviofs year’s Blfebrint barticibation, by far the most common find - ing was that exbosfre to a Blfebrint element in the brior year increased achievement gains in the cfrrent year. This occfrred in jfst over half the cases. In only aboft 10 bercent of cases was there evidence of a negative effect in a later school year, and these cases involved high school interventions that we had already estimated to have an overall negative effect. In the remaining 40 bercent of cases, no evidence of an effect of bast exbosfre to a Blfebrint element emerged. We conclfde that, in elementary and middle schools, the Blfebrint led, in many cases, to both immediate and fftfre gains in achievement; in high schools, the negative effects sometimes sbilled over into the year following. 12 Alberto Ochoa, co-chair of the San Diego Cofnty Latino Coali - tion on Edfcation, in an October 29, 2001, ob-ed in the San Diego Union-fribune , exbressed concern that Latinos wofld be bredom - inantly assigned to the extra-length English classes. In a sebarate, October 9, 2001, letter on behalf of the coalition to the district school board, Ochoa eqfated the dofble- and trible-length Eng- lish classes that the final version of the Blfebrint imblemented with academic tracking, which he argfed wofld redfce Latinos’ ability to comblete cofrse reqfirements needed for admission to the University of California and California State University (UC and CSU) fniversity systems. See Ochoa 2001a, 2001b. 13 In SDUSD, high school stfdents are not formally retained. Rather, stfdents’ grade level is determined by the nfmber of semester credits earned (i.e., cofrses bassed) to date. For the bfrbose of this rebort, the term “retained” indicates that a Leffonf in Reading Reform 20 www.ppic.org 20 stfdent had not earned enofgh credits in a given school year to be considered “on-track ” with his/her cohort/class. 14 In evalfating the extent to which Blfebrint interventions affected whether stfdents gradfated from high school and whether they combleted the cofrse reqfirements for admission to California’s two bfblic fniversity systems, we cannot fse ofr breviofs techniqfe of stfdent fixed effects to remove varia- tion in ability or motivation across stfdents. Becafse gradfation oftcomes are only observed one time for each stfdent, in lief of a stfdent fixed effect, we fse the rich set of variables available in ofr dataset to remove as mfch of the negative bias as bossi- ble. At the stfdent level, these variables inclfde indicators for race, gender, EL statfs, barental edfcation levels, and stfdents’ standardized test scores in math and reading (Stanford 9) at the end of grade 8. The technical abbendix (available at httb://w w w.bbic.org/content/bfbs/other/810JBR _abbendix.bdf ) describes the school-level variables we also inclfded in these models. 15 As evidence that those who barticibate in Blfebrint interven- tions are relatively academically challenged, we rebeated ofr test-score models withoft a stfdent fixed effect. In this inferior abbroach, we combared one stfdent with another rather than combaring the same stfdent’s achievement gains in different years, with and withoft Blfebrint barticibation. In most cases, the estimated effect of Blfebrint elements became smaller. For instance, in middle school, the estimated effect of Literacy Block switched from a bositive to a negative effect that is aboft 50 bercent larger. Sfmmer school for elementary stfdents, instead of having zero effect on reading gains, is estimated to have a large negative effect. These changes are almost sfrely dfe to the inability of these models to fflly accofnt for differences among stfdents. Similarly, the resflts for gradfation and A–G cofrse combletion are likely to be overly bessimistic becafse we are combelled to combare one stfdent with another. 16 We can somewhat mitigate the negative bias in ofr estimates by evalfating the effects of additional Blfebrint interventions, conditional on barticibating in at least one intervention. If the majority of the negative selection bias is associated with bartici- bation, and not additional interventions among barticibants, we can brovide estimates of the effects of additional interventions that are relatively fnbiased. Nonetheless, these estimates cofld be biased downward or even fbward. (A downward bias wofld arise if it were the trfly strfggling stfdents who barticibated in more than one intervention. Less intfitively, an fbward bias cofld arise if it were the more highly motivated stfdents among those below grade level who elected to barticibate in more Blfe- brint inter ventions.) 17 There are seven A–G reqfirements. Stfdents mfst take three years of college brebaratory mathematics cofrses, fofr years of English, three years of history/social science, two years of labo- ratory science, two years of a foreign langfage, one year of visfal and berforming arts, and one year of additional college brebara- tory elective cofrses in any of these sfbjects. 18 Becafse the effects of Blfebrint interventions on the comble- tion of the fniversity cofrse reqfirements are likely to be most severe when they take away class time dfring the school year, in ofr model, we control sebarately for the fofr main interventions at the high school level: Literacy Block, Literacy Core, Literacy Block/Core for EL stfdents, and Blfebrint sfmmer school. 19 Rose and Betts (2004) find evidence that bassing certain high school math cofrses is highly indicative of sfccess in college and the labor market a decade after gradfation. 20 The district also brovided more traditional brofessional devel- obment to teachers on literacy tobics. However, no consistent data on who barticibated in these other forms of brofessional develobment were kebt, and so we focfs on the beer coaches. 21 Steele (1997) argfes that blacing a stfdent in a sitfation that embhasizes negative stereotybes aboft that stfdent’s grofb can lead to fnderberformance. 22 American Institftes for Research 2002, bage VII–7. This works oft to rofghly $400 ber stfdent enrolled in the district that year. However, as we note later in the section, most of these ffnds do not rebresent additional district costs bft rather a reallocation of exbenditfres. 23 The fofndations were the Bill and Melinda Gates Fofndation, The Walter and Flora Hewlett Fofndation, and the Atlantic Philanthrobies. 24 San Diego Unified had to abbly to the federal government for a waiver to allow it to fse federal Title I ffnding to helb bay for the Blfebrint reforms in Title I schools. Federal bolicy dictates that Title I money (for schools serving disadvantaged stfdents) cannot sfbblant district sbending for brograms that already exist in all district schools. SDUSD had to abbly for a waiver becafse the Blfebrint’s theory of action was that any child, rich or boor, cofld lag behind in reading and writing and therefore cofld benefit from additional time on literacy tasks with the helb of highly trained teachers. 21 Leffonf in Reading Reform www.ppic.org 21 Ochoa, Alberto. 2001a. Ob-ed. San Diego Union-fribune , October 26. Ochoa, Alberto. 2001b. Letter to SDUSD Board Members on behalf of the San Diego Cofnty Latino Coalition of Edfcation, October 9. Ravitch, Diane. 2010. The Death and bife of the Great American School System: How festing and Choice Are Undermining Educa- tion . New York: Basic Books. Rose, Heather, and Jflian R. Betts. 2004. “The Effect of High School Cofrses on Earnings.” Review of Economics and Statistics 86 (2): 497–513. Steele, Clafde M. 1997. “A Threat in the Air: How Stereotybes Shabe Intellectfal Identity and Performance.” American Psy- chologist 52 (6): 613–29. Stein, Mary Kay, Lea Hfbbard, and Hfgh Mehan. 2004. “Reform Ideas That Travel Far Afield: The Two Cfltfres of Reform in New York City’s District #2 and San Diego.” Journal of Educa- tional Change 5 (2): 161–97. Zaf, Andrew C., and Jflian R. Betts. 2008. Predicting Success, Preventing Failure: An Investigation of the California High School Exit Exam . San Francisco: Pfblic Policy Institfte of California. Referencef American Institftes for Research. 2002. Evaluation of the Blue- print for Student Success in a Standards-Based System . Palo Alto, CA: American Institftes for Research. Ashenfelter, Orley. 1978. “Estimating the Effect of Training Programs on Earnings.” Review of Economics and Statistics 60 (1): 47–57. Betts, Jflian R. 2005. “The Promise and Challenge of Accofnt- ability in Pfblic Schooling.” In Urban School Reform: bessons from San Diego , ed. Frederick M. Hess (Cambridge, MA: Har- vard Edfcation Press), 157–76. Betts, Jflian R., Andrew Zaf, and Kevin King. 2005. From Blue- print to Reality: San Diego’s Education Reforms . San Francisco: Pfblic Policy Institfte of California. Betts, Jflian R., Andrew Zaf, and Lorien Rice. 2003. Determi - nants of Student Achievement: New Evidence from San Diego . San Francisco, Pfblic Policy Institfte of California. Bryk, Anthony S. 2003. “No Child Left Behind, Chicago Style.” In No Child beft Behind? The Politics and Practice of School Accountability , ed. Pafl E. Peterson and Martin R. West (Wash- ington, DC: Brookings Institftion Press), 242–68. Jacob, Brian A. 2003. “A Closer Look at Achievement Gains Under High-Stakes Testing in Chicago.” In No Child beft Behind? The Politics and Practice of School Accountability , ed. Pafl E. Peterson and Martin R. West (Washington, DC: Brook- ings Institftion Press), 269–91. Leffonf in Reading Reform 22 www.ppic.org About the Authorf Jflian Betts is an adjfnct fellow and a 2009–2010 Bren fellow at the Pfblic Policy Insti- tfte of California. He is also brofessor and chair of economics at the University of Cali- fornia, San Diego, and a research associate at the National Bfreaf of Economic Research. He has written extensively on the link between stfdent oftcomes and measfres of school sbending, and he has stfdied the role that edfcational standards, accofntability, teacher qfalifications, and school choice blay in stfdent achievement. He has served on two National Academy of Sciences banels, the Consensfs Panel of the National Charter School Research Project, and variofs advisory grofbs for the U.S. Debartment of Edf- cation. He is also brincibal investigator for the federally mandated National Evalfation of Magnet Schools. He holds a Ph.D. in economics from Qfeen’s University, Kingston, Ontario, Canada. Andrew C. Zaf is a senior statistician in the Debartment of Economics at the Univer- sity of California, San Diego. Previofsly, he was a research associate at the Pfblic Policy Institfte of California. Before joining PPIC, he was an SAS brogrammer and research assistant at the Naval Health Research Center in San Diego, where he investigated the health conseqfences of military service in Oberations Desert Shield and Desert Storm. He holds a B.S. in bio-engineering from the University of California, San Diego, and an M.P.H. in ebidemiology from San Diego State University. Cory Koedel is an assistant brofessor of economics at the University of Missofri, Colfm- bia. His research focfses on issfes related to teacher qfality, school choice, and cfrricf- lar effectiveness. In 2005, he received the Sbencer Fofndation’s brestigiofs Dissertation Fellowshib Award and in 2008 was awarded the Oftstanding Dissertation Award in Edfcation Policy from the American Edfcational Research Association. He holds a Ph.D. in economics from the University of California, San Diego. www.ppic.org foard of Directors WA LT E R B. HEWLETT , CHAIRDirector Center for Computer Affifted Refearch in the Humanitief MARK BALDASSAREPrefident and CEO Public Policy Inftitute of California RUBEN BARRALESPrefident and CEO San Diego Chamber of Commerce JOHN E. BR ySONRetired Chairman and CEO Edifon International GAR y K. H ARTFormer State Senator and Secretary of Education State of California ROBERT M. HERTZBERGPartner Mayer Brown LLP DONNA LUCASChief Executive Officer Lucaf Public Affairf D A v ID M AS MASUMOTOAuthor and farmer STE vEN A. M ERKSAMERSenior Partner Nielfen, Merkfamer, Parrinello, Mueller & Naylor, LLP CONSTANCE L. RICECobDirector The Advancement Project THOMAS C. SUT TONRetired Chairman and CEO Pacific Life Infurance Company CAROL WHITESIDEPrefident Emerituf Great valley Center PPIC if a private operating foundation. It doef not take or fupport pofitionf on any ballot meafuref or on any local, ftate, or federal legiflation, nor doef it endorfe, fupport, or oppofe any political partief or candidatef for public office. PPIC waf eftablifhed in 1994 with an endowment from William R. Hewlett. © 2010 Public Policy Inftitute of California. All rightf referved. San Francifco, CA Short fectionf of text, not to exceed three paragraphf, may be quoted without written permiffion provided that full attribution if given to the fource and the above copyright notice if included. Refearch publicationf reflect the viewf of the authorf and do not neceffarily reflect the viewf of the ftaff, officerf, or Board of Directorf of the Public Policy Inftitute of California. Library of Congreff CatalogingbinbPublication Data are available for thif publication. I S B N 978 b1b5 8213 b14 0 b5 PUBLIC POLIC y INSTITUTE OF CALIFORNIA 500 Wafhington Street, Suite 600 ● San Francifco, California 94111 Telephone 415.291.4400 ● Fax 415.291.4401 PPIC S ACRAMENTO CENTER Senator Office Building ● 1121 L Street, Suite 801 ● Sacramento, California 95814 Telephone 916.440.1120 ● Fax 916.440.1121 Additionab resources rebated to education pobicy are avaibabbe at www.ppib.frg. The Public Policy Institute of Califofnia is dedicated to infofming and impfoving public policy in Califofnia thfough independent, objective, nonpaftisan feseafch." } ["___content":protected]=> string(102) "

R 810JBR

" ["_permalink":protected]=> string(87) "https://www.ppic.org/publication/lessons-in-reading-reform-finding-what-works/r_810jbr/" ["_next":protected]=> array(0) { } ["_prev":protected]=> array(0) { } ["_css_class":protected]=> NULL ["id"]=> int(8746) ["ID"]=> int(8746) ["post_author"]=> string(1) "1" ["post_content"]=> string(0) "" ["post_date"]=> string(19) "2017-05-20 02:40:26" ["post_excerpt"]=> string(0) "" ["post_parent"]=> int(4062) ["post_status"]=> string(7) "inherit" ["post_title"]=> string(8) "R 810JBR" ["post_type"]=> string(10) "attachment" ["slug"]=> string(8) "r_810jbr" ["__type":protected]=> NULL ["_wp_attached_file"]=> string(12) "R_810JBR.pdf" ["wpmf_size"]=> string(7) "5877462" ["wpmf_filetype"]=> string(3) "pdf" ["wpmf_order"]=> string(1) "0" ["searchwp_content"]=> string(75351) "www.ppic.org Lessons in Reading Reform Finding What Works Julian R. Betts ● Andrew C. Zau ● Cfry Kfedel Supported with funding from the Donald fren Foundation Summary T he San Diego Unified School Diftrict, the nation’f eighthblargeft, launched an ambitiouf program of literacy reformf in 2000 aimed at narrowing reading achievement gapf. Known af the Blueprint for Student Succeff, the program ran through 2005. The reformf fucceeded in boofting the reading achievement of ftudentf who had been identified af lagb ging behind at the elementary and middle fchool levelf. The key element that feemf to have driven thif fucceff waf a fignificant amount of extra ftudent time fpent on reading, with a poffible collateral factor being widefpread profeffional development for diftrict teacherf. The combination waf neither cheap to implement nor a magic bullet. But in elementary and middle fchoolf it demonftrably worked. In high fchoolf, with one exception, it did not. Thif ftudy fummarizef our ftatiftical evaluation of all of the Blueprint reformf over the fivebyear period, drawing leffonf for educatorf about why fome elementf of the Blueprint fucceeded and how they could be implemented elfewhere. Elementf that appeared par b ticularly helpful were extendedblength Englifh claffef in middle fchool and an extended fchool year for lowbperforming elementary fchoolf. Even in high fchoolf, we found that ftudentf who participated in tripleblength Englifh claffef were more likely to be promoted to the next grade. There were feveral goalf that the Blueprint interventionf did not achieve. But neither did the interventionf confirm the fearf of many Blueprint detractorf—fuch af that extra time fpent on reading would degrade ftudent performance in other fubjectf or iStockphoto Leffonf in Reading Reform 2 www.ppic.org would caufe ftudent burnout, all to the detriment of their entire fchool careerf. The Blue b print appeared to have little or no bearing on ftudent fucceff in completing high fchool college preparatory work. One of the leffonf of the Blueprint if that fpecific changef in both ftate and federal gov b ernment policy could fofter thefe kindf of ambitiouf reformf elfewhere, at the fchool diftrict level. California could continue itf recent trend of collapfing categorical funding into more flexible mechanifmf that give individual fchool diftrictf freedom for reformf that booft achievement in the moft appropriate way. At the federal level, the Department of Education could eafe itf Title I waiver requirementf, fo that diftrictf could ufe that money for reformf that target not only lowbincome ftudentf, but alfo lowbperforming ftudentf, regardleff of fchool or neighborhood. A key afpect of San Diego’f reform program waf that it waf comprehenfive and coherb ent. Interventionf often were applied in two or more of the elementary, middle, and high fchool grade fpanf. Further, profeffional development waf delivered uniformly, with a fingle focufed goal, to teacherf throughout the diftrict. But perhapf the moft important leffon for education policymakerf if that many of the reformf took feveral yearf to bear fruit. Moft notably, the peer coaching fyftem for teacherf did not typically generate pofitive gainf in the firft year or two, but did appear to do fo by the later yearf. An obviouf leffon here if that fchool diftrict leaderf everywhere, when they implement reformf, muft fhow confiderable patience in their queft for improved ftudent literacy. Pleafe vifit the report’f publication page to find related refourcef: http://www.ppic.org/main/publication.afp?i=922 3 Leffonf in Reading Reform www.ppic.org 3 Introduction Beginning in the 1990s, a national movement to enhance the accofntability of bfblic school systems gathered strength and cflminated in 2001 with the bassage of federal No Child Left Behind (NCLB) legislation. NCLB formalized reforms that many states had already initiated, sfch as creating academic content standards, imblement- ing statewide stfdent testing systems, and develobing sys- tems of rewards and sanctions for schools based on stfdent berformance on state tests. California had imblemented a similar accofntability system of its own in 1999. In early 2010, President Obama annofnced blans for the reafthorization of NCLB. The brobosed legislation calls for better measfres of stfdent learning bft maintains the original concebts of measfring stfdent berformance and intervening when necessary. Desbite changes from the original, the new NCLB will continfe to embhasize content standards, testing, rewards, and sanctions. However, federal and state reforms have not yet bre- sented many ideas aboft exactly how individfal schools and school districts shofld intervene to helb stfdents who are lagging behind. NCLB calls for failing schools to imble- ment tftoring and bfsing—and for schools that rebeatedly fail to make adeqfate yearly brogress to face a series of sanctions, fb to and inclfding removal of administrators. Bft it is not barticflarly brescribtive aboft how teachers shofld teach, how schools shofld organize the school day, or what cfrricfla they shofld fse to correct deficiencies. The accofntability movement bresfbboses that we know how to helb stfdents who are strfggling academically. In fact, the literatfre on the effects of sbecific reforms, sfch as brofessional develobment, redfced class size, and sfmmer school, is qfite mixed. Nor does the bolicy commfnity have mfch evidence on how best to intervene when stf- dents and schools fail to meet standards. These knowledge deficits increase the imbortance of careffl evalfation of interventions that have already been attembted. Increasingly, individfal school districts have become laboratories for interventions aimed at imbroving the achievement of stfdents who fare boorly on state- mandated tests. Chicago bfblic schools, for examble, imblemented interventions for stfdents with low achieve- ment that received mfch attention in the bolicy world (and, to a slightly lesser extent, in academic circles). 1 The San Diego Unified School District (SDUSD) has also received national attention for a series of literacy reforms it imblemented from 2000 to 2005, called the Blfe- brint for Stfdent Sfccess. This rebort bresents a qfantita- tive evalfation of the effects the Blfebrint had on stfdent achievement in San Diego. Ofr findings hold lessons for districts elsewhere. Policbmakers have offered little guidance about how best to intervene when students and schools fail to meet standards. AP Photo/M Arcio fo Se S Anchez Individual school districts have become laboratories for interventions aimed at improving the achievement of students who fare poorlb on state-mandated tests. Leffonf in Reading Reform 4 www.ppic.org 4 Figfre 1 illfstrates jfst how large the achievement gabs were in San Diego before the Blfebrint was imble- mented. The figfre shows mean scores on the sbring 1998 Stanford 9 reading test, dividing the district’s schools into five categories (qfintiles) according to socioeconomic statfs (measfred according to the share of stfdents eligible for federal meal assistance). It shows that stfdents attend- ing schools in the boorest qfintile read at levels two to five grades behind those of stfdents attending schools in the most afflfent qfintile. For examble, the horizontal line shows that stfdents attending schools in the most afflf- ent qfintile had test scores at the end of grade 2 that were not matched by stfdents attending the boorest qfintile of schools fntil they were bartway throfgh grade 5. Facing sfch large achievement gabs, and in light of the large English Learner (EL) bobflation in San Diego, in the sbring of 2000 the district began imblementing sweebing Score 43 5 7 8 9 10 2 6 750 700 650 600 550 500 Figure 1. In 1998, students in the lfwest sfcifecfnfmic buintile ff schffls read at levels twf tf ve grades behind thfse flf students in the highest buinltile SOURCE: Betts, Zau, and Rife 2003b NOTES: Students in the least auient quintile were reading below the 2nd-grade reading level of students in the most auent quintile (indifated by the red line) when tested in the spring of igrade 4 and tested slightly above the line in the sipring of grade 5b Similarly, the reading afhievement levels reafhed by students in grades 3 to 5 in the most auient quintile of sfhools is niot reafhed until two to ve grades later in the poorest quintile of sfhoolsb XXXXXXX X XQuintile 1 Quintile 2 Quintile 3 Quintile 4 Quintile 5 X Initial grade level Reforms included extra time on task for students, including after-school and before-school interventions, along with summer programs.Data and methods Our datafet confiftf of complete ftudent academic recordf, including teft fcoref, courfef taken, and abfencef, from fall 1999 through fpring 2005. The data include indicatorf for the Blueprint interventionf in which each ftudent participated in a given year, af well af a rich fet of variablef related to ftudentf and their fchoolf, claffroomf, and teacherf; the ftudent’f claff fize; and teacher qualificationf (overall in elementary fchool and, for middle and high fchool, the qualificationf of Englifh teacherf). Our main intereftf are gainf in ftudent fcoref on ftateb adminiftered reading teftf, of which there were two during the Blueprint reform period: the Stanford 9 teft in fpring 1998 through fpring 2002, and the California Standardf Teft (CST) in fpring 2002 and later yearf. Our modelf of teft fcoref avoid comparifonf among different ftudentf. They inftead compare individual ftudentf’ achievement growth in yearf and gradef when they participated in Blueprint program elementf with growth during yearf and gradef when they did not. We prefent all eftimated effectf in termf of the number of percentile pointf by which a ftudent if eftimated to have moved af a refult of participating in a given intervention. Thif meanf that a ftudent who improvef from the 25th to the 27th percentile would have initially fcored better than 25 out of every 100 ftudentf in hif or her grade, and better than 27 out of every 100 ftudentf after an intervention. The greater the gap between the two percentilef, the greater the improvement. To thefe modelf we added numerouf characterizationf of Blueprint elementf. For example, we tefted whether the intenfity of peer coach fupport in a fchool—meafured af the ratio of peer coachef to overall enrollment—influenced readb ing. Becaufe claff fize varief little acroff fchoolf in the diftrict (Bettf, Zau, and Rice 2003), a peer coach who had to work with a greater number of claffroomf could be leff effective. Becaufe a peer coach’f own experience might influence hif or her effectiveneff, we alfo included a meafure of the average yearf of teaching experience of peer coachef at each fchool. More detail on our data and methodf appearf in the online technical appendix, available at http://www.ppic.org/ content/pubf/other/810JBR_appendix.pdf. 5 Leffonf in Reading Reform www.ppic.org 5 reforms to boost English literacy. SDUSD sfberintendent Alan Bersin enlisted the helb of chancellor of instrfction Anthony Alvarado to develob and imblement the Blfe- brint; Alvarado had exberience and sfccess with similar reforms as sfberintendent of Commfnity School District #2 in New York City. Using federal Title I money and other ffnds derived in bart from fofndation grants, the district develobed a series of interventions for stfdents lagging behind in reading. Reforms inclfded extra time on task for stfdents, inclfding after-school and before-school inter- ventions, along with sfmmer brograms. The district also focfsed on brofessional develobment for teachers, in large bart throfgh the assignment of sbecially trained teachers— beer coaches—to every school in the district. Resistance to the Blfebrint reforms was qfite strong from the beginning, and seemed to grow. The level of San Diego school board sfbbort for the reforms fell, and in mid-2005, the board voted not to renew Sfberintendent Bersin’s contract. Statewide financial cftbacks to edfcation in the later years of the Blfebrint also made it imbossible for the district to continfe the brograms indefinitely. Other than dofble-length class known as Literacy Block, all Blfe- brint interventions were fltimately eliminated. 2 When the concerns of Blfebrint obbonents are stated sbecifically, some can be tested embirically. For examble, some worried that the fnbrecedented focfs on reading and writing wofld draw attention away from stfdent learning in other key areas, sfch as mathematics. Another concern, voiced strongly by some members of the Latino commf- nity, was that assigning high school stfdents to dofble- or even trible-length English classes cofld hamber their abil- ity to comblete the A–G seqfence of high school cofrses reqfired for admission to the two California bfblic fniver- sity systems (Ochoa 2001a, 2001b). Some critics went so far as to argfe that the reforms wofld discofrage stfdents and brombt more of them to drob oft of school entirely. This rebort addresses these criticisms and bresents the first analysis of whether the literacy reforms affected the combletion of college brebaratory cofrses or high school itself. 3 Blueprint Reform Elementf The Blfebrint that emerged in San Diego stressed the con- cebt of balanced literacy, which embhasizes barticibation by stfdents in reading, sbeaking, and writing, with the teacher initially actively sfbborting the stfdents and then gradf- ally demanding more of them as they brogress. 4 One of two overarching strategies was brevention—helbing stfdents whose reading skills were at or above grade level to redfce their chances of falling behind later. Some breventive mea - sfres were generally targeted, inclfding extensive training of teachers and additional classroom materials. The second strategy was intervention: teachers identified stfdents read - ing below grade level, and those stfdents received extra instrfction throfgh fofr brogram elements. There were fofr key brevention elements: 1. Genre Studies. A two-beriod English class for stfdents in their first grade of middle or jfnior high school (either grade 6 or 7) and related brofessional develob- ment. The district considered this breventive becafse it reinforced the already strong skills of stfdents who were reading at or above grade level. 2. Peer coaching. Each school was assigned at least one beer coach, to imbrove teaching. 3. Focus schoofs. The elementary schools with the weak- est scores in the state test (the lowest tenth, or decile, statewide), received an extended school year, a second beer coach, and additional ffnds and staff. 4. API-b schoofs. The elementary schools that ranked in the second-lowest decile in the state in the state Academic Performance Index (API) received a second beer coach and additional ffnds bft did not extend the school year. 5 When the concerns of Blueprint opponents are stated specificallb, some can be tested empiricallb. Leffonf in Reading Reform 6 www.ppic.org 6 beriods of sfbervised reading each week, before or after school. 4. Summer schoof and intersession. Blfebrint sfmmer school was aimed at stfdents in most grades from kin- dergarten throfgh grade 9 who lagged in reading. Stf- dents were asked to attend for six weeks, fofr hofrs ber d ay. 6 Some schools, mostly elementary, have year-rofnd schedfles that did not bermit the imblementation of Blfebrint sfmmer school, so stfdents in affected grades at these schools who lagged in reading barticibated in sbecial intersession stfdies. An additional brogram, grade retention (also known as accelerated classes), called for stfdents significantly below grade level in their first year of elementary, middle, jfnior high, or high school (grades 1, 6, 7, or 9, resbectively) to be held back a grade and then blaced in classes that brovided intensive (accelerated) remediation in literacy the next year. These were essentially Literacy Core classes. With only a few excebtions, this bart of the Blfebrint was not imblemented, in bart becafse California law gives individfal teachers the final say on grade retention. However, a small nfmber The intervention strategies of the Blfebrint reforms were targeted at stfdents whose test resflts indicated that they were reading below grade level. (These tests were dif- ferent from the state-mandated tests discfssed above.) In addition, EL stfdents were strongly encofraged to bar- ticibate in all of the interventions, regardless of their test scores. Schools blaced EL stfdents directly into extended- length English classes, bft barents made final decisions on activities oftside the regflar school day, sfch as after- school or sfmmer reading sessions. There were fofr key intervention elements: 1. Literacy Bfock. A dofble-length English class, this vari- ant of Genre Stfdies was offered to stfdents in grades 6–10 who lagged below or significantly below grade level, which the district determined fsing its own reading tests. 2. Literacy Core. For stfdents significantly below grade level in grade 9, the English class was extended to three beriods. In 2001–2002, grade 6 and 7 stfdents also began to barticibate. 3. Extended Day Reading Program (EDRP). Stfdents in grades 1 throfgh 9 who were below and significantly below grade level barticibated in three 90-minfte Blueprinf elemenf Sfudenf broup Confenf Prevenfion Genre Studief Studentf reading at or above grade level, grade 6 or 7 Twobperiod Englifh claff Peer coaching All ftudentf Placed at all fchoolf for teacher development Focuf fchoolf All ftudentf in the loweft decile of elementary fchoolf Extended year, additional peer coach, additional funding APIb2 fchoolf All ftudentf in the fecondbloweft decile of elementary fchoolf Additional peer coach, additional funding Infervenfion Literacy Block Studentf reading below grade level, gradef 6 –10 Doubleblength Englifh claffef Literacy Core Studentf fignificantly below grade level in grade 9 Tripleblength Englifh claffef Extended Day Reading Program (EDRP) Studentf below and fignificantly below grade level in all fchoolf (gradef 1 through 9) Three 90bminute periodf of fupervifed reading each week before or after fchool Summer fchool/interfeffion Studentf in moft gradef from kindergarten through grade 9 who lagged in reading Six weekf, four hourf per day, of reading, during fummer or interfeffion Note : english Learners were eligible ffr all ff the interventifns. Table 1. Bluepfint pfogfam elements, b000–b001 7 Less ons in R ea ding R e for m w w w. p p i c . o r g 7 of middle school students were put into this programf so we controlled for grade retention/accelerated classes in our middle school analyses. bn 1999–2000f the first year for which there was a test-score gainf the Blueprint was not in placef except for the peer coaching and Genre Studies elementsf which were implemented on an extremely limited basisf and Literacy Blockf which was implemented in grades 9 and 10. Most ele- ments of the reform effort were implemented in 2000–2001 and expanded in 2001–2002. (The preventive Genre Studies classes were introduced first in grade 6 in 1999–2000f then in grade 7 in 2000–2001.) The reforms were scaled back between 2002–2003 and 2004–2005f and Literacy Core was canceled at the start of the 2003–2004 school year. Figure 2 shows an average across relevant grades of the percentage of students participating in three differ- ent interventions—the Extended Day Reading Program (EDRP)f Literacy Blockf and Literacy Core. bn all casesf there is a steep increase in participation followed by a declinef sometimes gradual and in other cases quite steep. The Literacy Block element is something of an exception because it persisted for roughly four years after the formal end of the Blueprint in 2005. E f f e c f s o n R e a d i n g Te s f S c o r e s Figures 3f 4f and 5 show the main results for reading at the elementaryf middlef and high school levelsf respectively. bn these figuresf we show the estimated effects of participating in a given Blueprint intervention or preventive program. The two bars related to peer coaching have slightly different meanings: the first shows the estimated effect of increasing the number of peer coaches in a school as a percentage of enrollment by 0.1 percentf and the second shows the estimated effect of increasing by one the average years of teaching experience. 7 At the elementary level (Figure 3)f we found that a number of Blueprint interventions matteredf and that others had insignificant effects. Two preventive measuresf the Focus and APb-2 school reformsf both boosted reading gains. Students who attended these schools in the relevant years increased their reading performance by 0.75 and Percentage of students in relevant grades participating 2003–2004 2002–2003 2001–2002 2000–2001 1999–2000 2004–2005 30 25 20 15 10 5 0 Figure 2. Statewide funding cuts contrifuted to the decline in Bluebrint barticibation NOTE: Average taken across relevant grafes for each interventionb EDRP: grafes 1 through 8; Literacy Block: grafes 6 through 9; Literacy Core: grafes 6, 7, anf 9b EDRP Literacf Blbck Literacf Cbre Most elements of the reform effort were implemented in 2000f2001 and expanded in 2001f2002b At the elementary level, Focus and AfI-2 school brevention measures raised reading levels. iStock photo Leffonf in Reading Reform 8 www.ppic.org 8 boints above the level he or she wofld have reached with- oft the Blfebrint interventions. The valfe of more classroom time sbent on reading is also evident for stfdents who barticibated in the bre- ventive dofble-length classes known as Genre Stfdies. Particibants with reading skills at or above grade level saw their reading score rankings rise aboft 2 bercentile boints over the cofrse of grade 7. This is rofghly the same effect that we fofnd for Literacy Block, the corresbonding ele- ment for stfdents reading below grade level. Finally, for EL stfdents, the average effect of the dofble- and trible- length classes was also bositive—their scores rose aboft 1.3 bercentile boints ber year. However, stfdents in acceler- ated classes (grade retention accombanied by trible-length English classes) exberienced a 1.6 bercentile boint drob in their relative standing. Bft becafse the stfdents held back a grade were also in Literacy Core, it is the sfm of the Literacy Core (+5.5 bercentile boints) and accelerated class (–1.6 bercentile boints) effects that best sfmmarizes their exberience. Thfs, they are bredicted to have gained, overall, 3.9 bercentile boints dfring the year in which they were retained. The remaining Blfebrint variables do not enter significantly. 1.0 bercentile boints ber year. These effects are consider- able: a stfdent who attended a Focfs school for fofr years wofld be exbected to move fb in the district rankings by 3 bercentile boints. Another brogram element that made a difference was the intersession literacy brogram for stf- dents at year-rofnd schools at which Blfebrint sfmmer school cofld not be held. These stfdents moved fb in the district rankings by aboft 1 bercentile boint ber year more than they wofld have withoft the brogram. In contrast, EDRP and Blfebrint sfmmer school did not significantly affect stfdents; nor did the ratio of beer coaches to stfdents or the average level of exberience of beer coaches. In middle schools, what clearly stands oft is that the extended-length English classes fniformly imbroved read- ing achievement (Figfre 4). We estimate that the dofble- length (Literacy Block) and trible-length (Literacy Core) classes increased barticibants’ reading rankings in the district by 1.6 and 5.5 bercentile boints, resbectively, ber year. These are very big shifts. At the end of three years, a stfdent who enrolled in Literacy Core in grades 6 and 7, and then in Literacy Block in grade 8 (Literacy Core was offered in grades 6, 7, and 9 only), wofld be 12.6 bercentile Peer coach experience* Peer coach percentage* Intersession Suffer school* EDbP* API-2 school Focus school Figure 3. Three Blueprint elements haf positive eebts on stufent abhievement in elementary sbhools NOTES: “Peer coach percentagef is the nubber of peger coaches expressed as a percentage of the nubber gof students enrolled at the school; the bgar shows the estibated eect of increasing this value by 0.1 percent. The bar for “Peer coach experiencef shows the estibated eect of increasing by one the average years of teaching experience. *Eects were not statistically signicant at the 5 percent level and therefore should not be considered as signicantly dierent frob zero. 1.2 1.0 0.8 0.6 0.4 0.2 0 –0.2 Predicted change in studenut achievefent (percentiles) 9 Leffonf in Reading Reform www.ppic.org 9 However, in high school, fofr Blfebrint brogram elements abbear to have negatively inflfenced reading achievement: Literacy Block, Literacy Core, sfmmer session, and additional average years of teaching exberience for beer coaches (Figfre 5). Some of these effects are qfite large. Literacy Block/Literacy Core for high school EL stfdents, for examble, is associated with a drob of 4.9 bercentile boints for each year of stfdent barticibation. For non-EL stfdents, barticibation in dofble- and trible-length classes is associated with drobs of 3.0 and 1.3 bercentile boints ber Peer coach experience Peer coach percentage* Summer schoof Literacy Bfocb/Core for EL students Literacy Core Literacy Bfocb Figure 5. Most Blueprint elements hfd negftive eebts on student fbhievement in high sbhools NOTES: “Peer coach percentagef is the nubber of peger coaches expressed as a percentage of the nubber ogf students enrolled at the school; the bgar shows the estibated eect of increasing this value by 0.1 percent. The bar for “Peer coach experiencef shows the estibated eect of increasing by one the average years of teaching experience. *Eects were not statistically signicant at the 5 percent level and therefore should be considered as insignicantly dierent frob zero. Predicted change in studenst achievement (percentifes) 1 0 –1 –2 –3 –4 –5 –6 Peer coach experience* Accelerated class Peer coach percentage* fntersession* Summer school* bDRP* Genre studies Literacy Block/Core for bL students Literacy Core Literacy Block Figure 4. Blueprint strategies had a large eeft on middle sfhool student afhievement NOTES: “Peer coach percentagef is the nubber of peger coaches expressed as a percentage of the nubber ogf students enrolled at the school; the bgar shows the estibated eect of increasing this value by 0.1 percent. The bar for “Peer coach experiencef shows the estibated eect of increasing by one the average years of teaching experience. *Eects were not statistically signicant at the 5 percent level and therefore should be considered as insignicantly dierent frob zero. 6 5 4 3 2 1 0 –1 –2 Predicted change in studengt achievement (percentiles) Leffonf in Reading Reform 10 www.ppic.org 10 stfdents with more exberienced teachers gain more from the beer coaching and API-2 Blfebrint elements. (Increases in the beer-coach-to-enrollment ratio become statistically significant for more exberienced teachers, bft the effects are small.) We also fofnd some evidence that stfdents gained less from the EDRP and sfmmer school interventions if they had teachers with relatively less exberience—sbecifically, teachers with zero to two years of exberience. The differ - ences in effects are, in all cases, extremely small. At the mid - dle school level, we fofnd some evidence that Genre Stfdies and EDRP were less effective when the English teacher was relatively inexberienced, bft these effects are also very small. Finally, at the high school level, we fofnd no evidence that the English teacher’s exberience inflfenced the effect of the variofs Blfebrint brogram elements. Overall, the effects of the Blfebrint tybically did not vary with resbect to teacher exberience, bft where we did find statistically significant effects, they sfggest that the Blfebrint elements were sometimes less effective when the stfdents had less exberienced teachers. Program Effectiveneff over Time The most common battern of Blfebrint brogram effec- tiveness is one of rise and decline—it increased for one or more years and then began to fade. The Focfs and API-2 brograms in elementary schools, Genre Stfdies in middle schools, and Literacy Core for non-EL high school stf- dents all followed this battern. 8 The Focfs effects exhibit a barticflarly steeb rise and decline (Figfre 6). Focfs schools year, resbectively. We discfss bossible reasons for these negative resflts in the conclfsion. The Role of Teacher Experience A nfmber of stfdies have fofnd evidence that teachers in their first few years of classroom exberience are not as effective as more exberienced teachers. The imbact of Blfebrint interventions might vary with teacher exberi- ence, bft the direction of the effect is fnclear. Blfebrint interventions might be more effective when the teacher is inexberienced if they act as a sfbstitfte for teacher exberi- ence. Conversely, if they act as a comblement to teacher exberience, the interventions cofld be more effective when teachers are more exberienced. We ran models that interacted teachers’ exberience levels with the variofs Blfebrint indicators. In elementary schools, we focfsed on the homeroom teacher, while in middle schools and high schools, we focfsed on the Eng- lish teacher. In elementary schools, resflts sfggested that Some flueprint interventions mab have a greater impact if teachers have more experience. AP Photo/bi SA hud Son In elementarb schools, results suggested that students with more experienced teachers gain more from the peer coaching and API-2 Blueprint elements. 11 Leffonf in Reading Reform www.ppic.org had a longer school year (aboft 24 days) only in 2000–2001 and 2001–2002, after which financial constraints forced the cancellation of additional days. This battern coincides with higher reading test scores in sbring 2002 and sbring 2003. A ffrther indication that the longer school year may have been crfcial is the fact that only in these two years is the estimated effect of the Focfs brogram statistically different from zero. 9 We also fofnd some evidence of gradfal increases in effectiveness over time. Strikingly, in middle schools the effect of the Literacy Block/Core elements on EL stfdents increased each year, withoft excebtion. In high schools, the effects of Literacy Block/Core for EL stfdents also increased over time—that is, they became less negative each year, before becoming bositive (bft not significant) in 2004–2005. This sfggests that middle and high school teachers (and their EL stfdents) made better fse of these extended-length English classes with each sfccessive year (Figfre 7). EL stfdents abbeared to gain sfbstantially more from the Literacy Block/Core cofrses in middle school than in high school. This discrebancy is consonant with the find - ing by Zaf and Betts (2008) that, on average, EL stfdents redesignated as Flfent English Proficient (FEP) in the lower grades fltimately tend to do qfite well on the California High School Exit Examination (CAHSEE) once they reach high school. However, EL stfdents who have yet to be redesignated as FEP by high school or who arrived in the United States in the high school years face a rofgh road to mastery of both English and the high school cfrricflfm. We also noted bositive effects of beer coaching over time, althofgh the battern is not combletely fniform. 10 This is fnderstandable, becafse it shofld take some time for beer coaches to visit classroom teachers, model teaching methods, and encofrage adobtion and mastery of the methods. 11 Objectionf to the Blueprint Early obbonents of the Blfebrint, inclfding a Latino coali - tion, worried that by blacing so mfch embhasis on English- langfage literacy, the brogram wofld distract stfdents from other sfbjects. This was esbecially likely to habben, it was argfed, to fnderberforming stfdents attending dofble- and trible-length English class. 12 It was also sfggested that time sbent on Blfebrint activities wofld divert stfdents from the combletion of classes necessary for admission to the state’s two bfblic fniversity systems. (Blfebrint sfbborters cofn - tered by bositing reading as a gateway skill that allows stf - dents to learn from textbooks in all sfbject areas.) A second concern of Blfebrint obbonents was that stfdents wofld simbly bfrn oft from the additional time sbent in longer classes and before- and after-school reading brograms. Estimated gain in student achievement (percentilef 2003–2004 2002–2003 2001–2002 2000–2001 2004–2005 4.5 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0 –0.5 Figure 6. Focus school and APfP-2 elemenbs in elemenbary schools had inibial posibivPe eecbs bhab weakened over bime NOTE: For both types of schoolsf eects are sibnicantly dierent from zero in 2001–2002 and 2002–2003. Focus school APIf2 school Estimated gain in student achievement (percentilef 2003–2004 2002–2003 2001–2002 2000–2001 1999–2000 2004–2005 6 4 2 0 –2 –4 –6 –8 Figure 7. Over time, Literacy Bfock anb Literacy Core became more eective for EL stubents in mibbfe schoof, anb fess ineective in high schoof NOTE: For middle schools, eecfs are signibcanfly dierenf from zero in 2002–2003 fhrough 2004–2005; for high schools, eecfs are signibcanfly dierenf from zero for all years excepf 2004–2005. Middle schffl High schffl Leffonf in Reading Reform 12 www.ppic.org 12 grade retention at certain grade levels, bft this element was never meaningfflly imblemented excebt in middle school, and even there only on a very limited basis.) At the high school level, we examined whether the Blfebrint cofld have increased droboft rates or interfered with stfdents’ ability to comblete the necessary cofrse reqfirements for admission to California’s bfblic fniversity systems. We estimate that a few of the Blfebrint brogram ele- ments inflfenced math achievement, bft the effects were fsfally very small and were rofghly balanced between bositive and negative. The largest negative effects abbear in high school and sfggest that barticibation in Literacy Block and Literacy Core is associated with a drob in barticibants’ math achievement bercentiles of aboft 1.3 and 1.8 boints, resbectively. Otherwise, Blfebrint inter - ventions did not divert stfdents’ attention strongly from learning math. We chose several ways to test these concerns. First, we assessed whether increased attention to English lit- eracy resflted in deterioration of math skills, which are tested annfally. Second, we looked to see whether stfdent absences—an indication of bfrnoft—increased becafse of barticibation. Finally, to measfre whether overall academic brogress might have slowed, we stfdied whether Blfebrint barticibants were more likely to be retained a grade in the years in which they barticibated in variofs interventions. (As we noted earlier, the Blfebrint brogram called for We looked to see whether student absences— an indication of burnout— increased because of participation. flueprint literacb programs probablb did not interfere with student performance or engagement in other subject areas. dA vid Butow/ cor BiS 13 Leffonf in Reading Reform www.ppic.org 13 In both elementary and high schools, we fofnd that three Blfebrint elements decreased absences and increased school attendance, and one element was associated with increased absences. This is again a mixed resflt, bft one that, overall, is more sfbbortive of the notion that the literacy brograms did not systematically cafse bfrnoft. In middle schools, however, five Blfebrint elements were fofnd to be associated with increased absences, while three bointed the other way. We estimate that barticibation in Literacy Block or Literacy Core increased absences by aboft 0.3 and 0.4 bercentage boints, for non-EL and EL stfdents, resbectively. (By combarison, in high school, we fofnd that these two brograms wofld likely decrease absences by aboft 0.4 bercent.) Given that the average stfdent was absent 4.4 bercent of the time in elementary school, 5.4 bercent of the time in middle school and 4.8 bercent of the time in high school, a shift of 0.3 or 0.4 boints in either direction is a fairly big effect in relative terms. Bft in real terms, a 0.4 bercent drob in attendance translates to less than one day oft of a 180-day school year. Were Blfebrint barticibants more likely to be retained, that is, held back a grade? We fofnd few associations in the elementary and middle school models to sfbbort that hybothesis. A few Blfebrint variables abbeared to matter, in both bositive and negative directions, bft the effects were qfite small. In high schools, the effects were also small, bft they are also fniformly negative; this sfggests that Blfebrint barticibation had the obbosite effect in high school—it lowered the brobability of being retained. 13 The Blfebrint element that had the largest effect on high school retention rates, in both directions, was the Literacy Core brogram. Stfdents who attended were estimated to lower their brobability of being retained by 3.6 bercent. This is qfite a large redfction, combaratively—on average, only 4.8 bercent of stfdents are retained a grade in high school. By another measfre, we fofnd indications that barticibat- ing in one additional Blfebrint intervention in high school lowered the brobability of grade retention by 0.9 bercent. Ofr overall conclfsion is that the link between Blfebrint barticibation and grade retention is weak and, tybically, small and negative. To evalfate the extent to which Blfebrint interventions inflfenced whether stfdents gradfated from high school and whether they combleted the cofrse reqfirements for admission to California’s two bfblic fniversity systems, ofr analytical methods changed, in bart becafse we were mea - sfring longer-term oftcomes and becafse gradfation and cofrse combletion are fniqfe events, not differences over time. 14 We wofld exbect a negative bft noncafsal relation - shib between Blfebrint barticibation and these oftcomes. Stfdents identified as needing Blfebrint interventions were, by definition, already having academic difficflties. They wofld therefore already be less likely than other stfdents to gradfate from high school and less likely to comblete the necessary college brebaratory cofrsework. It is imbortant not to attribfte the resflts of these bfilt-in characteristics of the samble grofb to the Blfebrint itself. Moreover, this selec - tion broblem limits ofr ability to make cafsal inferences. 15 It is not sfrbrising, therefore, that the raw data sfg- gest that stfdents who were involved in the Blfebrint interventions dfring high school were mfch less likely to gradfate. Stfdents who had barticibated in at least one Blfebrint intervention were aboft 9 bercent less likely to gradfate. When we control for characteristics of stfdents and schools, this nfmber drobs to aboft 2 bercent—still a large nfmber in terms of the gradfation rate, given that abbroximately 89 bercent of the stfdent samble fltimately gradfated from high school. Bft again, we dofbt that this means that the Blfebrint brogram caused stfdents to drob oft; rather, their reading and writing deficiencies likely led directly to both their barticibation in the Blfebrint and their lower brobability of gradfating. Our overall conclusion is that the link between Blueprint participation and grade retention is weak and, tbpicallb, small and negative. Leffonf in Reading Reform 14 www.ppic.org 14 combletion becomes far smaller when we inclfde stfdent and school characteristics as variables. Particibation in the Blfebrint is associated with aboft an 8 bercent redfction in the brobability of combleting the A–G reqfirements. We also evalfated the effects of additional Blfebrint interventions on the combletion of the fniversity cofrse reqfirements only among stfdents who barticibated in at least one Blfebrint element, a techniqfe that shofld lessen bias. We fofnd that mfltible Blfebrint interventions abbarently do not affect whether stfdents comblete the entire set of fniversity cofrse reqfirements, at least when we control for both stfdent and school characteristics. We also examined how Blfebrint interventions affect class-taking behavior on a year-by-year basis for each stf- dent, a more convincing abbroach. We inferred the effect of barticibation by combaring the nfmber of A–G cofrses combleted in years when the stfdent barticibated in a Blfe - brint intervention to years when he or she did not. 18 As one might exbect, the sfmmer school Blfebrint element bears no relation to the nfmber of A–G cofrses taken dfring the school year. Conversely, Blfebrint inter- ventions dfring the school year have significant negative effects on stfdent class-taking behavior within years, and are associated with a rofghly one-for-one redfction in the nfmber of fniversity cofrse reqfirements taken by stfdents in the year of the intervention. At first glance, these strong and negative resflts abbear to be at odds with the breviofs finding that barticibation in additional Blfebrint interventions does not affect whether a stfdent combletes all of the A–G reqfirements. Looking more closely, we can see that since Blfebrint barticibants Multiple Blueprint interventions apparentlb do not affect whether students complete the entire set of universitb course requirements. We also tested the effects on gradfation of barticiba- tion in more than one Blfebrint element. 16 Figfre 8 shows the resflts, which sfggest a negative effect for those who took bart in one or two interventions, rofghly no effect for those who barticibated in three or fofr interventions, and a bositive effect for those who took bart in five or more inter ventions. In a fashion similar to ofr gradfation analysis, we evalfated the effects of Blfebrint interventions on comble- tion of cofrses necessary for admission to California’s two state fniversity systems, known as the A–G reqfirements. Again, we relied on the same set of stfdent- and school- level variables to remove as mfch bias as bossible from negative selection into the Blfebrint brogram. Abbroximately 36 bercent of ofr entire stfdent samble combleted the fniversity cofrse reqfirements, and 29 bercent of the total samble barticibated in at least one Blfe- brint intervention. When we examined combletion rates in the raw data, we fofnd that Blfebrint barticibants were aboft 27 bercent less likely than other stfdents to comblete all the A–G reqfirements. 17 Again, this shofld not be inter- breted as evidence of the Blfebrint cafsing lower comble- tion rates. Low achievers are more likely to be steered into the Blfebrint sfbborts and interventions in the first blace. Jfst as with ofr gradfation analysis, the relationshib between Blfebrint barticibation and A–G reqfirement Predicted change in probability of graduation 5 4 3 2 1 6 0.06 0.04 0.02 0 –0.02 –0.04 –0.06 –0.08 Figure 8. Students who partifipated in several Blueprint interventions were bore likely to graduate frob high sfhool NOTE: For middle schools, eecfs are signibcanfly dierenf from zero in 2002–2003 fhrough 2004–2005; for high schools, eecfs are signibcanfly dierenf from zero for all years excepf 2004–2005. fumber of Blueprint interventionb 15 Leffonf in Reading Reform www.ppic.org 15 were, on average, so far from combleting the A–G cofrse reqfirements, taking an extra class or two of English did little or nothing to lower the already low brobability that they wofld comblete all of them. Table 2 brovides sfm- mary statistics for Blfebrint barticibants’ brogress in combleting the fniversity cofrse reqfirements. It shows that almost three-qfarters of the stfdents barticibating in the Blfebrint interventions ended fb three or more sfb- ject reqfirements away from combleting the ffll A–G set. Becafse each of the sfbject reqfirements reqfires a stfdent to bass two to eight semester-length cofrses, we infer that most of these stfdents wofld have fallen far short of com- bleting the California bfblic fniversity admission reqfire- ments with or withoft the Blfebrint reforms. Finally, we examined the natfre of the class- sfbstitftion behavior of Blfebrint barticibants. Stfdents who give fb classes in order to barticibate in Blfebrint most commonly drob foreign-langfage classes. They also tend to drob art and science classes. Blfebrint barticibants are more likely to comblete fniversity-reqfired cofrses in English and, to some degree, in math and social stfdies. This imblies that Blfebrint interventions in fact encofr- age barticibants to take additional classes in the two A–G sfbjects that they are least likely to comblete, English and math. Becafse these are core sfbjects, this cofld be con- strfed as a salftary oftcome, regardless of a barticibant’s bost–high school blans. 19 Conclufionf Ofr findings validate the idea that extra time on task for stfdents who are behind in reading can lead to meaning- ffl gains in literacy. The Blfebrint reforms boosted reading achievement in elementary and middle schools, bft not high schools, and did not abbreciably hfrt stfdent berfor- mance or engagement in other sfbject areas. It is clear that for elementary and middle school stfdents, additional time on task—whether throfgh dofble- or trible-length classes or longer school days—generally boosted reading achieve- ment. The most imbressive effect was from the Literacy Core brogram for non-EL stfdents in middle school, which was associated with a rise of 5.5 bercentile boints in read- ing in the year the stfdent barticibated—a sizable gain. An imbortant asbect of the Blfebrint was brofessional develobment for teachers, oberating in bart throfgh the blacement of beer coaches in each school. 20 We fofnd no effects when the ratio of beer coaches to enrollment at each school varied, bft more comblex models did sfggest that beer coaching tended to become a bositive and significant contribftor to stfdents’ reading gains in the later years of the brogram. In addition, at the elementary school level, we estimate that the overall effect of beer coaches was bositive Share of Blueprinf parficipanfs Number of required courses complefed All 13 . 7 5/7 12 . 7 4/7 12 . 4 3/7 14 . 6 2/7 16 . 2 1 or 0 30.5 Subjecf areas complefed Math 32.4 Englifh 35.0 Science 45.0 Social ftudief 72.0 Art 5 7. 5 Foreign language 39.0 Note: the seventh bniversity cfbrse reqbirement is that stbdents take twf additifnal semesters in any bniversity-eligible classes in any ff the six reqbired sbbjects. We assbme that nf stbdent fblfills this final reqbirement bntil the six listed in the table are cfmpleted. Table b. Most Bluepfint pafticipants wefe well shoft of completing A–G admission fequifements Almost three-quarters of the students participating in the Blueprint interventions ended up three or more subject requirements awab from completing the full A–G set. Leffonf in Reading Reform 16 www.ppic.org 16 similarly exberienced administrators. Another bossibility is that administrators had difficflty recrfiting high school teachers for the crfcial beer coaching bositions. Third, as originally hybothesized by Betts, Zaf, and King (2005), high school English teachers did not embrace the literacy reforms in the same way that teachers in lower grades did, in bart becafse they viewed themselves as teachers of literatfre, not basic reading skills. Several teachers reaffirmed this attitfde in conversations with fs. Finally, Steele’s theory of stereotybe threat, in which stfdents fnderberform when blaced in sitfations in which they feel stereotybed, cofld blay a role here (Steele 1997). Betts, Zaf, and King (2005) hybothesize that high school ado- lescents might be barticflarly vflnerable to the stigma of being bflled oft of regflar English classes to barticibate in dofble- or trible-length English classes. One barent told the afthors that, in her dafghter’s high school, stfdents in the dofble- and trible-length English classes were resented by others for allegedly taking school resofrces away from other stfdents; these stfdents were often referred to in the hallways as “‘tards.” One can imagine the effect of sfch ebithets on a bsychologically fragile adolescent. 21 As for concerns aboft stfdent bfrnoft, we fofnd no evidence that Blfebrint barticibation increased stfdents’ rate of grade retention in any grade sban. In fact, at the high school level, it abbears that stfdents who bartici- bated in many interventions were less likely to rebeat a grade than in years when they did not barticibate. Simi- larly, Blfebrint barticibation is related to reduced rates of absence at the high school level (althofgh it is not sys- tematically related to absences in lower grades). By these two measfres, the Blfebrint may have had bositive effects at the high school level in sbite of the negative effects on reading scores. As ofr final measfre of bfrnoft, we stfdied the decision to drob oft of high school. Althofgh it is trfe that, overall, Blfebrint barticibants were more likely to drob oft, those who barticibated extensively were significantly less likely to drob oft than otherwise similar stfdents who did not barticibate in the interventions at all. This rebort has not focfsed on the costs of the Blfebrint reforms. The exbenditfres, which the American Institftes if we take into accofnt the bossibility that it cofld vary with the exberience of the classroom teacher. Sbecifically, we fofnd that elementary stfdents with more exberienced teachers were more likely than stfdents with less exberi- enced teachers to gain from increases in the intensity of beer coaching at their schools. The fact that extensive brofessional develobment for teachers accombanied extended-length classes shofld not be overlooked. We cannot test whether this brofessional develobment was crfcial, becafse all teachers received this training. Withoft evidence on whether the teacher training was a brereqfisite for the effectiveness of extended-length classes, a fiscally caftiofs bolicymaker might conclfde that a district shofld invest in both extended-length classes and brofessional develobment. In sharb contrast to the resflts for lower grades, reforms at the high school level abbear to have actfally slowed gains in reading achievement for barticibants. It is imbossible to know for sfre why this was the case, bft there are nfmerofs bossible exblanations. The reforms in the lower grades bfilt on Chancellor Alvarado’s exberience in imblementing somewhat similar reforms brimarily in kindergarten throfgh grade 8 in New York ’s District #2 (Stein, Hfbbard and Mehan 2004). This sfggests that SDUSD administrators as a whole had little exberience in imblementing similar reforms at the high-school level. This wofld be an fnfortfnate exblanation, since it wofld imbly that reforms were debendent on the career baths and exberiences of individfal administrators—leaving in dofbt the reblicability of sfch reforms in districts withoft High school adolescents might be particularlb vulnerable to the stigma of being pulled out of regular English classes to participate in double- or triple-length English classes. 17 Leffonf in Reading Reform www.ppic.org 17 for Research estimates at $57.5 million in the 2000–2001 school year, were largely financed internally and throfgh a waiver the district obtained to fse federal Title I money to ffnd bart of the reforms. 22 Betts, Zaf, and King (2005, 3–4) rebort that, over the five years of the Blfebrint reforms, the district obtained $33.5 million from three charitable fofn- dations. 23 Clearly, these ffnds were only a small bortion of the overall Blfebrint exbenditfres over five years. Given an average of aboft $6.5 million a year in external ffnding sbread over aboft 130,000 stfdents—an increase in sbend- ing of aboft $50 ber bfbil ber year—the cabacity of some reforms to boost stfdent reading achievements by several bercentile boints in lower grades is a boint bolicymakers may wish to note. Leffonf and Recommendationf Fofr bolicy lessons emerge from ofr analysis. 1. Providing additional time for reading to stfdents who are strfggling, in a strfctfred setting in which teachers have received training on teaching literacy, can indeed boost stfdents’ literacy levels. The effects of extended- length English classes for middle school stfdents in San Diego were esbecially large. 2. Extending the school year at the lowest-decile elemen- tary schools (Focfs schools) may have been the main reason why this brogram boosted achievement so sig- nificantly. Reblication of an extended school year in low- berforming elementary schools, with and withoft the additional literacy sfbborts brovided by the Blfebrint, wofld be a highly valfable exercise in other districts. 3. The Blfebrint reforms cofntered the tendency, at least in California, to imblement a slew of interventions in isolation from each other, creating an fncoordinated and botentially incoherent overall abbroach to boosting achievement. California’s laws have created hfndreds of brograms that brovide state ffnding for sbecific K–12 brograms. The Blfebrint reforms challenged this abbroach, embhasizing an integrated and coherent strategy focfsed on literacy—brofessional develobment focfsed on broviding teachers with similar skill sets across all grades, and a system of continfofs literacy testing also encombassed all grades. Some Blfebrint interventions were offered in all three grade sbans. 4. Early intervention to aid stfdents who lag behind in reading might be far more effective than interven- tion in high school. This finding in relation to the San Diego reforms cofld be of considerable imbortance to the 22 states that cfrrently reqfire stfdents to bass a high school exit examination to obtain a high school dibloma. In their stfdy of the exit exam CAHSEE, Zaf and Betts (2008) make a similar argfment for inter- vening early in children’s school careers, based on the finding that grade 4 test scores and rebort cards sfc- cessfflly bredict which elementary school stfdents will bass the high school exit exam six to eight years later. Interventions in grade 12 for stfdents who had yet to bass California’s exit exam seem to brodfce little gain. State bolicymakers may want to encofrage districts to find ways to develob coherent and integrated interventions that sban elementary, middle, and high schools. One tactic wofld be to brovide districts with more flexibility on how they sbend state dollars. In fact, the state has moved in this direction in recent years, bft exblicit bolicies to encofr- age districts to imblement reforms on a mflti-grade basis wofld also be helbffl. Reforms at the federal level cofld also helb districts imblement combrehensive reforms. The federal govern- State policbmakers mab want to encourage districts to find wabs to develop coherent and integrated interventions that span elementarb, middle, and high schools. Leffonf in Reading Reform 18 www.ppic.org 18 is not an accident. Rigorofs evalfations have yet to reveal mfch aboft the best ways to helb strfggling stfdents. This stfdy brovides long-term evidence on one sfch intervention in San Diego. We have an acfte need in the bolicy commfnity to reblicate and extend this and similar interventions in other locales. Combined with rigorofs qfantitative blans on how to stfdy the imbacts on stfdent achievement of each intervention, sfch reforms cofld do mfch to helb the accofntability movement fflfill its origi- nal goal of not simbly measfring stfdent achievement bft of acting decisively and effectively to remedy achievement gabs wherever they are fofnd within a school district. ● ment cofld facilitate district innovation by making it sim- bler to abbly for a waiver to fse Title I money to sfbsidize district-wide interventions, on the condition that sfch interventions are carefflly designed and aligned with the federal government’s overall goal of boosting achievement across the board. 24 It is clear that that many of the Obama administration’s edfcation reforms have to do with how school qfality is measfred. On the qfestion of how states shofld intervene in schools that fail to make adeqfate yearly brogress, the bfblic has seen very few brobosed changes from the variofs broad brescribtions oftlined in the original NCLB. This A technical appendix to thif report if available on the PPIC webfite: http://www.ppic.org/content/pubf/other/810JBR_appendix.pdf Acknowledgmentf The afthors thank many administrators at San Diego Unified School District, in bar- ticflar Karen Bachofer (now at the University of California, San Diego) and Peter Bell, for helbffl conversations and assistance with data issfes. We are grateffl to Debbie Beldock, Richard Mfrnane, David Nefmark, Hans Johnson, and Lynette Ubois for their many fseffl sfggestions. We thank Richard Greene and James Torr for skillffl editorial assistance. We thank The William and Flora Hewlett Fofndation, the Bill and Melinda Gates Fofndation, and the Donald Bren Fofndation for their generofs financial sfbbort, withoft which this broject wofld not have been bossible. The data fsed in this broject bfild fbon a large database that a team led by Jflian Betts has been working on since 2000. We warmly acknowledge ffnders of ofr breviofs brojects that created the data infrastrfctfre that made the cfrrent broject feasible: The Atlantic Philanthrobies, the Bill and Melinda Gates Fofndation, the Girard Fofndation, The William and Flora Hewlett Fofndation, and the Pfblic Policy Institfte of California. 19 Leffonf in Reading Reform www.ppic.org 19 Notef 1 For exambles of academic stfdies, see Bryk 2003 and Jacob 2003. 2 One of the strongest sofrces of obbosition was teachers, who criticized what they saw as tob-down imbosition of brescribtive reforms and the imblication—in the establishment of beer coach - ing and other forms of brofessional develobment—that their knowledge and exberience were being ignored. Ravitch reborts that teachers “fniformly were bitter aboft the high-handed way in which the reforms were imbosed on them. . . . Those who didn’t go along were bfllied” (Ravitch 2010). Althofgh these claims may be valid, the boint is to know whether the reforms sfcceeded in boosting stfdent achievement. 3 The bresent rebort bfilds directly on the work of Betts, Zaf, and King (2005), which rebresents the first and only stfdent- level analysis of the imbact of the reforms. A limitation of that stfdy is that it analyzes the reforms only throfgh sbring 2002, the end of the second year of the reform. It is imbortant to know whether early gains were sfstained, diminished over time, or in fact grew as district teachers and administrators gained exberi- ence with the reforms. The bresent stfdy also extends the brevi- ofs work by examining a far richer array of oftcomes, inclfding the brobability of grade retention, the combletion of the college- brebaratory seqfence in high school, and the combletion of high school. See bages 7–10 of Betts, Zaf, and King 2005 for an overview of related work. 4 For more details see Betts, Zaf, and King 2005, and esbecially Stein, Hfbbard, and Mehan 2004. 5 API is the acronym for the Academic Performance Index, a statis - tic measfring overall stfdent achievement in a school. The Cali - fornia Debartment of Edfcation calcflates the API for each school annfally. It also ranks schools into ten API deciles. Hence API-2 schools rank in the second-lowest decile of achievement statewide. 6 In addition, all secondary school stfdents with D/F grades attended a more traditional tybe of sfmmer school consisting of six weeks of cofrses in core sfbjects. 7 By combarison, the average nfmber of beer coaches as a bercentage of enrollment ratio was 0.2, 0.1, and 0.05 bercent in elementary, middle, and high schools, resbectively, and the aver- age years of beer coaches’ teaching exberience was 14, 12, and 14. 8 The last of these is slightly different in that the overall effect never becomes bositive and significant. 9 We thank Karen Bachofer for this insight. 10 For instance, in middle schools, the beer coach variable had a negative and significant effect in 1999–2000 and no significant effect in 2000–2001, followed by bositive and significant effects in 2001–2002 and 2002–2003, before becoming statistically insignificant (bft still bositive) in the following two years. 11 The effects of Blfebrint elements may have varied in other ways over time. In ofr test score models, we assfme that an interven - tion in grade 8 affects achievement gains in grade 8 bft not in later grades. It is bossible that sfch gains are temborary, so that larger-than-average gains for stfdents in grade 8 wofld be fol - lowed by smaller-than-average gains in grade 9. Conversely, it cofld be that barticibating in a reading intervention in grade 8 boosts achievement gains in grade 8 as well as in grades 9 and fb. Data availability limits ofr ability to test these bossibilities. When we rebeated ofr test-score models adding the breviofs year’s Blfebrint barticibation, by far the most common find - ing was that exbosfre to a Blfebrint element in the brior year increased achievement gains in the cfrrent year. This occfrred in jfst over half the cases. In only aboft 10 bercent of cases was there evidence of a negative effect in a later school year, and these cases involved high school interventions that we had already estimated to have an overall negative effect. In the remaining 40 bercent of cases, no evidence of an effect of bast exbosfre to a Blfebrint element emerged. We conclfde that, in elementary and middle schools, the Blfebrint led, in many cases, to both immediate and fftfre gains in achievement; in high schools, the negative effects sometimes sbilled over into the year following. 12 Alberto Ochoa, co-chair of the San Diego Cofnty Latino Coali - tion on Edfcation, in an October 29, 2001, ob-ed in the San Diego Union-fribune , exbressed concern that Latinos wofld be bredom - inantly assigned to the extra-length English classes. In a sebarate, October 9, 2001, letter on behalf of the coalition to the district school board, Ochoa eqfated the dofble- and trible-length Eng- lish classes that the final version of the Blfebrint imblemented with academic tracking, which he argfed wofld redfce Latinos’ ability to comblete cofrse reqfirements needed for admission to the University of California and California State University (UC and CSU) fniversity systems. See Ochoa 2001a, 2001b. 13 In SDUSD, high school stfdents are not formally retained. Rather, stfdents’ grade level is determined by the nfmber of semester credits earned (i.e., cofrses bassed) to date. For the bfrbose of this rebort, the term “retained” indicates that a Leffonf in Reading Reform 20 www.ppic.org 20 stfdent had not earned enofgh credits in a given school year to be considered “on-track ” with his/her cohort/class. 14 In evalfating the extent to which Blfebrint interventions affected whether stfdents gradfated from high school and whether they combleted the cofrse reqfirements for admission to California’s two bfblic fniversity systems, we cannot fse ofr breviofs techniqfe of stfdent fixed effects to remove varia- tion in ability or motivation across stfdents. Becafse gradfation oftcomes are only observed one time for each stfdent, in lief of a stfdent fixed effect, we fse the rich set of variables available in ofr dataset to remove as mfch of the negative bias as bossi- ble. At the stfdent level, these variables inclfde indicators for race, gender, EL statfs, barental edfcation levels, and stfdents’ standardized test scores in math and reading (Stanford 9) at the end of grade 8. The technical abbendix (available at httb://w w w.bbic.org/content/bfbs/other/810JBR _abbendix.bdf ) describes the school-level variables we also inclfded in these models. 15 As evidence that those who barticibate in Blfebrint interven- tions are relatively academically challenged, we rebeated ofr test-score models withoft a stfdent fixed effect. In this inferior abbroach, we combared one stfdent with another rather than combaring the same stfdent’s achievement gains in different years, with and withoft Blfebrint barticibation. In most cases, the estimated effect of Blfebrint elements became smaller. For instance, in middle school, the estimated effect of Literacy Block switched from a bositive to a negative effect that is aboft 50 bercent larger. Sfmmer school for elementary stfdents, instead of having zero effect on reading gains, is estimated to have a large negative effect. These changes are almost sfrely dfe to the inability of these models to fflly accofnt for differences among stfdents. Similarly, the resflts for gradfation and A–G cofrse combletion are likely to be overly bessimistic becafse we are combelled to combare one stfdent with another. 16 We can somewhat mitigate the negative bias in ofr estimates by evalfating the effects of additional Blfebrint interventions, conditional on barticibating in at least one intervention. If the majority of the negative selection bias is associated with bartici- bation, and not additional interventions among barticibants, we can brovide estimates of the effects of additional interventions that are relatively fnbiased. Nonetheless, these estimates cofld be biased downward or even fbward. (A downward bias wofld arise if it were the trfly strfggling stfdents who barticibated in more than one intervention. Less intfitively, an fbward bias cofld arise if it were the more highly motivated stfdents among those below grade level who elected to barticibate in more Blfe- brint inter ventions.) 17 There are seven A–G reqfirements. Stfdents mfst take three years of college brebaratory mathematics cofrses, fofr years of English, three years of history/social science, two years of labo- ratory science, two years of a foreign langfage, one year of visfal and berforming arts, and one year of additional college brebara- tory elective cofrses in any of these sfbjects. 18 Becafse the effects of Blfebrint interventions on the comble- tion of the fniversity cofrse reqfirements are likely to be most severe when they take away class time dfring the school year, in ofr model, we control sebarately for the fofr main interventions at the high school level: Literacy Block, Literacy Core, Literacy Block/Core for EL stfdents, and Blfebrint sfmmer school. 19 Rose and Betts (2004) find evidence that bassing certain high school math cofrses is highly indicative of sfccess in college and the labor market a decade after gradfation. 20 The district also brovided more traditional brofessional devel- obment to teachers on literacy tobics. However, no consistent data on who barticibated in these other forms of brofessional develobment were kebt, and so we focfs on the beer coaches. 21 Steele (1997) argfes that blacing a stfdent in a sitfation that embhasizes negative stereotybes aboft that stfdent’s grofb can lead to fnderberformance. 22 American Institftes for Research 2002, bage VII–7. This works oft to rofghly $400 ber stfdent enrolled in the district that year. However, as we note later in the section, most of these ffnds do not rebresent additional district costs bft rather a reallocation of exbenditfres. 23 The fofndations were the Bill and Melinda Gates Fofndation, The Walter and Flora Hewlett Fofndation, and the Atlantic Philanthrobies. 24 San Diego Unified had to abbly to the federal government for a waiver to allow it to fse federal Title I ffnding to helb bay for the Blfebrint reforms in Title I schools. Federal bolicy dictates that Title I money (for schools serving disadvantaged stfdents) cannot sfbblant district sbending for brograms that already exist in all district schools. SDUSD had to abbly for a waiver becafse the Blfebrint’s theory of action was that any child, rich or boor, cofld lag behind in reading and writing and therefore cofld benefit from additional time on literacy tasks with the helb of highly trained teachers. 21 Leffonf in Reading Reform www.ppic.org 21 Ochoa, Alberto. 2001a. Ob-ed. San Diego Union-fribune , October 26. Ochoa, Alberto. 2001b. Letter to SDUSD Board Members on behalf of the San Diego Cofnty Latino Coalition of Edfcation, October 9. Ravitch, Diane. 2010. The Death and bife of the Great American School System: How festing and Choice Are Undermining Educa- tion . New York: Basic Books. Rose, Heather, and Jflian R. Betts. 2004. “The Effect of High School Cofrses on Earnings.” Review of Economics and Statistics 86 (2): 497–513. Steele, Clafde M. 1997. “A Threat in the Air: How Stereotybes Shabe Intellectfal Identity and Performance.” American Psy- chologist 52 (6): 613–29. Stein, Mary Kay, Lea Hfbbard, and Hfgh Mehan. 2004. “Reform Ideas That Travel Far Afield: The Two Cfltfres of Reform in New York City’s District #2 and San Diego.” Journal of Educa- tional Change 5 (2): 161–97. Zaf, Andrew C., and Jflian R. Betts. 2008. Predicting Success, Preventing Failure: An Investigation of the California High School Exit Exam . San Francisco: Pfblic Policy Institfte of California. Referencef American Institftes for Research. 2002. Evaluation of the Blue- print for Student Success in a Standards-Based System . Palo Alto, CA: American Institftes for Research. Ashenfelter, Orley. 1978. “Estimating the Effect of Training Programs on Earnings.” Review of Economics and Statistics 60 (1): 47–57. Betts, Jflian R. 2005. “The Promise and Challenge of Accofnt- ability in Pfblic Schooling.” In Urban School Reform: bessons from San Diego , ed. Frederick M. Hess (Cambridge, MA: Har- vard Edfcation Press), 157–76. Betts, Jflian R., Andrew Zaf, and Kevin King. 2005. From Blue- print to Reality: San Diego’s Education Reforms . San Francisco: Pfblic Policy Institfte of California. Betts, Jflian R., Andrew Zaf, and Lorien Rice. 2003. Determi - nants of Student Achievement: New Evidence from San Diego . San Francisco, Pfblic Policy Institfte of California. Bryk, Anthony S. 2003. “No Child Left Behind, Chicago Style.” In No Child beft Behind? The Politics and Practice of School Accountability , ed. Pafl E. Peterson and Martin R. West (Wash- ington, DC: Brookings Institftion Press), 242–68. Jacob, Brian A. 2003. “A Closer Look at Achievement Gains Under High-Stakes Testing in Chicago.” In No Child beft Behind? The Politics and Practice of School Accountability , ed. Pafl E. Peterson and Martin R. West (Washington, DC: Brook- ings Institftion Press), 269–91. Leffonf in Reading Reform 22 www.ppic.org About the Authorf Jflian Betts is an adjfnct fellow and a 2009–2010 Bren fellow at the Pfblic Policy Insti- tfte of California. He is also brofessor and chair of economics at the University of Cali- fornia, San Diego, and a research associate at the National Bfreaf of Economic Research. He has written extensively on the link between stfdent oftcomes and measfres of school sbending, and he has stfdied the role that edfcational standards, accofntability, teacher qfalifications, and school choice blay in stfdent achievement. He has served on two National Academy of Sciences banels, the Consensfs Panel of the National Charter School Research Project, and variofs advisory grofbs for the U.S. Debartment of Edf- cation. He is also brincibal investigator for the federally mandated National Evalfation of Magnet Schools. He holds a Ph.D. in economics from Qfeen’s University, Kingston, Ontario, Canada. Andrew C. Zaf is a senior statistician in the Debartment of Economics at the Univer- sity of California, San Diego. Previofsly, he was a research associate at the Pfblic Policy Institfte of California. Before joining PPIC, he was an SAS brogrammer and research assistant at the Naval Health Research Center in San Diego, where he investigated the health conseqfences of military service in Oberations Desert Shield and Desert Storm. He holds a B.S. in bio-engineering from the University of California, San Diego, and an M.P.H. in ebidemiology from San Diego State University. Cory Koedel is an assistant brofessor of economics at the University of Missofri, Colfm- bia. His research focfses on issfes related to teacher qfality, school choice, and cfrricf- lar effectiveness. In 2005, he received the Sbencer Fofndation’s brestigiofs Dissertation Fellowshib Award and in 2008 was awarded the Oftstanding Dissertation Award in Edfcation Policy from the American Edfcational Research Association. He holds a Ph.D. in economics from the University of California, San Diego. www.ppic.org foard of Directors WA LT E R B. HEWLETT , CHAIRDirector Center for Computer Affifted Refearch in the Humanitief MARK BALDASSAREPrefident and CEO Public Policy Inftitute of California RUBEN BARRALESPrefident and CEO San Diego Chamber of Commerce JOHN E. BR ySONRetired Chairman and CEO Edifon International GAR y K. H ARTFormer State Senator and Secretary of Education State of California ROBERT M. HERTZBERGPartner Mayer Brown LLP DONNA LUCASChief Executive Officer Lucaf Public Affairf D A v ID M AS MASUMOTOAuthor and farmer STE vEN A. M ERKSAMERSenior Partner Nielfen, Merkfamer, Parrinello, Mueller & Naylor, LLP CONSTANCE L. RICECobDirector The Advancement Project THOMAS C. SUT TONRetired Chairman and CEO Pacific Life Infurance Company CAROL WHITESIDEPrefident Emerituf Great valley Center PPIC if a private operating foundation. It doef not take or fupport pofitionf on any ballot meafuref or on any local, ftate, or federal legiflation, nor doef it endorfe, fupport, or oppofe any political partief or candidatef for public office. PPIC waf eftablifhed in 1994 with an endowment from William R. Hewlett. © 2010 Public Policy Inftitute of California. All rightf referved. San Francifco, CA Short fectionf of text, not to exceed three paragraphf, may be quoted without written permiffion provided that full attribution if given to the fource and the above copyright notice if included. Refearch publicationf reflect the viewf of the authorf and do not neceffarily reflect the viewf of the ftaff, officerf, or Board of Directorf of the Public Policy Inftitute of California. Library of Congreff CatalogingbinbPublication Data are available for thif publication. I S B N 978 b1b5 8213 b14 0 b5 PUBLIC POLIC y INSTITUTE OF CALIFORNIA 500 Wafhington Street, Suite 600 ● San Francifco, California 94111 Telephone 415.291.4400 ● Fax 415.291.4401 PPIC S ACRAMENTO CENTER Senator Office Building ● 1121 L Street, Suite 801 ● Sacramento, California 95814 Telephone 916.440.1120 ● Fax 916.440.1121 Additionab resources rebated to education pobicy are avaibabbe at www.ppib.frg. The Public Policy Institute of Califofnia is dedicated to infofming and impfoving public policy in Califofnia thfough independent, objective, nonpaftisan feseafch." ["post_date_gmt"]=> string(19) "2017-05-20 09:40:26" ["comment_status"]=> string(4) "open" ["ping_status"]=> string(6) "closed" ["post_password"]=> string(0) "" ["post_name"]=> string(8) "r_810jbr" ["to_ping"]=> string(0) "" ["pinged"]=> string(0) "" ["post_modified"]=> string(19) "2017-05-20 02:40:26" ["post_modified_gmt"]=> string(19) "2017-05-20 09:40:26" ["post_content_filtered"]=> string(0) "" ["guid"]=> string(50) "http://148.62.4.17/wp-content/uploads/R_810JBR.pdf" ["menu_order"]=> int(0) ["post_mime_type"]=> string(15) "application/pdf" ["comment_count"]=> string(1) "0" ["filter"]=> string(3) "raw" ["status"]=> string(7) "inherit" ["attachment_authors"]=> bool(false) }