Donate
Independent, objective, nonpartisan research

Reforming English Pathways at California’s Community Colleges, Technical Appendix

Database

This is the content currently stored in the post and postmeta tables.

View live version

object(Timber\Post)#3742 (44) { ["ImageClass"]=> string(12) "Timber\Image" ["PostClass"]=> string(11) "Timber\Post" ["TermClass"]=> string(11) "Timber\Term" ["object_type"]=> string(4) "post" ["custom"]=> array(5) { ["_wp_attached_file"]=> string(20) "0218mcr-appendix.pdf" ["wpmf_size"]=> string(6) "275078" ["wpmf_filetype"]=> string(3) "pdf" ["wpmf_order"]=> string(1) "0" ["searchwp_content"]=> string(27878) "Reforming English Pathways at California’s Community Colleges Technical Appendices CONTENTS Appendix A. Data and Methods Appendix B. Tables Marisol Cuellar Mejia, Olga Rodriguez, Hans Johnson, and Bonnie Brooks Supported with funding from the Bill and Melinda Gates Foundation, the College Futures Foundation, an d the Sutton Family Fund PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 2 Appendix A. Data and Methods Data This report uses both quantitative and qualitative data to achieve two goals : (1) to describe developmental English course offerings, student enrollment (where available), and outcomes across California’s 114 community colleges, and (2) to evaluate the effectiveness of one of t he most prevalent reforms that colleges are implementing , namely one -semester acceleration. Our quantitative approach utilizes student -level longitudinal data from the California Community College Chancellor’s Office Management Information System (COMIS). The dataset includes students enrolled across the 114 community colleges that comprise the California Community College system, and includes information on student demographics, course -taking behavior, course elements (e.g. , title of course, basic-skills status, minimum/maximum number of credits, etc.), and student outcomes (grades, credits earned, etc.). Table B 1 lists the variables from the COMIS data that we use in our analyses. Data was also collected from an exhaustive scan of college catalogs, websi tes, and other institutional documents. For the purposes of this report, we restrict the universe of developmental English courses to those that are required in order to take college composition. We used C -ID designations to identify college composition courses across colleges. Once we identified the college composition course in each college, we looked for it in the course description section of the college catalog and proceeded backward through the pre -requisites to the lowest level of remedial course wo rk offered by each college that was part of the pathways to college composition. This process allowed us to determine the type of sequence (IRW, separate reading and writing, only writing, or mixed sequence), the number of levels below transfer, and the number of courses required. It also allowed us to identify which colleges are offering one -term accelerated courses and/or co -requisite models. To determine the type of alternative sequence that best characterizes the college, we relied on both our exhaustive scan of college catalogs and actual enrollment data from the MIS data. Classifying courses/sequences in each one of the four types mentioned above—in particular, determining if a course was integrated or not —was challenging. Courses were classif ied as integrated or not based on their name and description. We coded courses as IRW if the course title or description use d the terms “academic reading & writing ,” “critical reading ,” etc. If course titles and descriptions only marginally mentioned readi ng, or if they use a skill s-based description (e.g. vocabulary, reading comprehension, etc.) , we coded them as only writing. To inform our quantitative results, we collected information from interviews with faculty and administrators from California ’s comm unity colleges. We selected interviewees from colleges that (1) had reform pathways with student throughput rates that were significantly better than those of the traditional English pathway and (2) were representative of each alternative English pathway w e studied: the integrated reading and writing pathway and one -term acceleration, with particular attention to the scale of the reform efforts. Out of 24 people we contacted, 13 were interviewed from 11 California community colleges: 12 English faculty and 1 administrator. We spoke with each interviewee for about one hour over the phone. Interviewees were asked a variety of questions pertaining to their background, how students enroll in and learn about alternative English pathways, student experiences in al ternative English pathways, aspects of the alternative pathway course (e.g. , motivation for offering the pathway, course structure , and curriculum), professional development, a s well as funding and scaling up English pathways. Open -ended questions were use d to facilitate the collection of information based on questions that do not restrict the participants’ opinions (Creswell and Plano Clark 2011). PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 3 The data collection and data analysis were carried out simultaneously to avoid the collection of repetitive an d unfocused data (Merriam 1998). Particularly, after each interview was conducted, researchers debriefed, reviewed detailed data notes and audio recordings , and kept notes to capture reflections, emerging themes, and points that needed to be pursued furthe r. This process of review and reflection informed all subsequent interviews. In this manner, data collected from each interview was continuously assessed and informed future interviews until data collection was complete. The data was also organized and cod ed on a secure spreadsheet. This approach was used to come up with a number of patterns and themes. Empirical Strategy for Estimating the Overall Effect of Acceleration in Developmental English on Student Success We focus our analysis on first -time studen ts in the academic year s 2014 –15 and 2015–16, who enrolled in developmental English before or on the fall term of 2016. We restrict our sample to (1) students with valid social security numbers and birthdays, (2) first -time students age 15 –64 at time of entry into the California community college system who (3) took an English or math course in their community college career, and (4) were only enrolled in either the accelerated course or a multilevel developmental English sequence. We exclude students who hold bachelor’s or associate degrees at time of entry , students enrolled in only summer or winter terms, and dual -enrollment students. After implementing these restrictions, our sample of interest include d 69,774 first-time students in the CCC system (15 % of which were enrolled in an accelerated course). To test the hypothesis that students enrolled in accelerated courses are more likely to enroll and succeed in college composition than students that went through a multilevel sequence, we use probit and ordinary least square fixed- effects specifications. Students who took the accelerated courses are selected into the treatment group. Our control group is any student who started the developmental sequence two levels below transfer. While there are a few differences between treatment and control groups, student characteristics are similar on the whole, which allows us to fairly compare these groups in our analyses. Our study focuses on four key outcomes: completion of developmental English, enrollment in college composition, successful completion of college composition (throughput), and total credits earned. Our regression models take the following form , where i indexes students and g colleges: ���� ��������= ��������+ ���� PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 4 course prior to college composition. In other words, we exclude from our treatment group any student who took one or more courses as part of the multilevel sequence before enrolling in the accelerated pathway. However, this model may generate biased estimates of ���� . The most notable arises due to omitted variable bias. Even when we control for a number of student characteristics, it is very difficult to measure other characteristics such as motivation or student ability upon college entry. Because these measures are likely correlated with enrollment in developmental education and success in college, even when students have similar observed characteristics, the inability to control for motivation and pr ior ability could lead to biased estimates. In addition, selection bias may arise due to differential enrollment patterns in developmental education. For instance, if the most motivated students are more likely to enroll in required developmental courses w hile the least motivated are more likely to delay or never enroll in developmental coursework, we can observe upward bias. We also run separate models that include interaction terms to evaluate the effect of acceleration by student attributes (race, gender , low-income status) and by college. The college- interacted model is especially noteworthy, as it assures our results are not solely driven by a single college. Comparing Our Estimates to Those in the Existing Literature Our study of accelerated developmental English in California’s community colleges uses data from over 700,000 students who first enrolled in any community college during the 2014– 15 and 2015–16 academic years. Using multivariate regressions that control fo r a variety of student, college, and course characteristics, we found that students who take a one -term accelerated developmental English course are substantially more likely to go on and complete a college- level English course than students who take the t wo-level sequence. These results hold by gender and by ethnic group; however, we find that equity gaps remain. The findings in our study are consistent with the evidence found by prior studies on one -term acceleration in the state of California and elsewhere in the nation. One of the earliest pieces of descriptive evidence on one- term acceleration in California comes from Chabot College. Looking at cohorts going back as far as 2000, Hern (2011) found that students enrolling in the open- access one- term accelerated pathway had completed college English at significantly higher rates within three years than students who started in a non- accelerated two-term pathway (54% vs. 28% for the fall 2008 cohort). Furthermore, students from all racial/ethnic groups exper ienced gains under the accelerated pathway; however, the gaps between racial/ethnic groups remained and in some instances widened. The overall findings of this research were subsequently corroborated in a quasi -experimental study of acceleration at Chabot by Edgecombe and colleagues (2014). The analysis drew from the data of 3,853 accelerated English students and 4,757 non- accelerated English students that enrolled for the first time at Chabot College between 1999 and 2010. The study employed propensity sco re matching (PSM) and regression analyses and controlled for student demographic and academic background to examine academic outcomes up to five years after entry. Overall, the research found that participation in the one -term acceleration course was posit ively associated with college English completion, credit accumulation, grade point average, certificate and degree attainment and transfer; but in some instances the gains for accelerated students diminished over time. Specifically, in the PSM model a 25 percentage point gain in transfer English completion within one year declined to 22 percentage points after five years. Furthermore, the PSM model found that the gains in transfer -ready status and graduation declined or became non -significant after five yea rs. However, the advantage students who took accelerated courses experienced in terms of grades and credit accrual was maintained over this timeframe. This study did not explore whether the impacts of acceleration varied for different racial/ethnic or gend er groups. PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 5 Our findings are also in line with Hayward and Willett’s (2014) study that examined the impact of accelerated developmental English pathways at ten community colleges in California. These pathways, implemented using the curricular and pedagogic al principles developed by the California Acceleration Project (CAP), provide an alternative to the traditional multilevel sequence composed of separate reading and writing courses. The accelerated pathway is intended to reduce the time students spend in developmental English and make the skills students learn more relevant to the work they will be expected to do in college -level English. The analysis drew from the data of 1,836 accelerated English students and 22,354 traditional English students enrolled a t 10 colleges during the 2011 –12 academic year and tracked outcomes for two years. The study employed a multivariate logistic regression with student background controls and found that the odds of completing college -level English for accelerated English st udents were 1.5 times higher than for students in the traditional English sequence. Importantly, they found that the odds varied significantly by college —colleges that granted open access to a one- term accelerated English approach (high acceleration) achieved odds of 2.3, compared to odds of 1.2 at colleges that required students to complete a pre -requisite course (low acceleration). Additionally, the study found that the accelerated approach improved outcomes for minority students, but gaps remained. The r esearch provides among the most promising evidence of developmental English reform available in the state of California and this has led to a surge in colleges adopting this reform. Research by Hodara and Jaggars (2014) also sheds light on the impact of ac celeration in the form of offering a shorter (one -term) versus a longer (two -term) developmental writing sequence. This research drew from the data of 3,932 accelerated English students and 7,498 non- accelerated English students that enrolled for the first time at a City University of New York community college between 2001 and 2007. The study employed propensity score matching and controlled for student background and cohort fixed effects to examine academic outcomes within three and five years after entry . The researchers found that students who started in a shorter, one -term writing sequence were 9.7 percentage points more likely to enroll in college English than their peers who started in a two - term sequence; students in the shorter sequence were also 6 percentage points more likely to complete college English within three years. However, this overall completion increase comes at a cost, when condition on enrollment in the college English course, accelerated students were found to pass the college English course at a lower rate (2.5 percentage points lower). Still, when examining longer -term outcomes, accelerated students outperformed their nonaccelerated peers. Namely, students who enrolled in the shorter writing sequence earned about two more credits ove r three years and earned a degree at a higher rate after five years (2.2 percentage points higher) than their peers who started in a longer writing sequence. While this study did not explore differences by race/ethnic or gender groups, it does shed light on the promise of developmental education in a very large and diverse urban community college system (over 70% of students are Latino and African American). While this evidence base suggests that accelerating student progression through developmental English by providing a one -term developmental English pathway has both positive early impacts and potential for longer - term positive impact, it is less clear whether the strategies in fact help close equity gaps for diverse groups of students. Also, while the ov erall evidence on one-term acceleration is promising, it is important to acknowledge that a newer approach to acceleration known as co- requisite remediation (discussed in detail in the main report) has also gained much momentum due to positive early findings (Cho et al. 2012; Coleman 2015; Denley 2016; Jaggars, et al. 2015; Jenkins et al. 2010). Overall, the mounting evidence on the benefits of acceleration has spurred adoption of reforms at colleges across California. Our review of the research suggests mo re work is needed to explore whether the impact of acceleration strategies persists beyond the first year and whether they help address equity gaps. PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 6 Caveats While our analysis employs rigorous statistical analysis and includes a substantial number of con trols, there are considerations to take into account with regard to the data we use and our methods. Our data does not contain information pertaining to student assessment scores, placement recommendation s, or high school GPA, all of which would more accurately control for student s’ academic record prior to enrolling in developmental English. We use students ’ first course in the developmental English sequence as a proxy variable. We track student outcomes for anywhere between 1 and 10 terms. While the sampl e sizes we do include in our study are sufficient to draw conclusions from our regression analyses, our analysis w ould have benefited from having larger sample sizes of students enrolled in accelerated pathways. One reason for having smaller -than- desired sample sizes is the recent implementation of these accelerated pathways. Our analysis focuses on enrollment in accelerated pathways in relation to student outcomes. However, other types of reforms were implemented within our timeframe of interest that may also have contribute d to our results, most notably assessment and placement reforms such as multiple measures and adjustments to the cut scores used to place students. Including college fixed effects in our model aims to account for these other reforms implemented at the college level, outside of alternative math pathways. PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 7 Appendix B. Tables In each of our regression models, we compare developmental education students in accelerated pathways to students who are in non- accelerated pathways that begin tw o levels below college composition. We develop three separate sets of models for the following dependent variables: (1) enrollment in college composition; (2) successful completion of college composition (also known as throughput); and (3) number of trans fer units completed. Table B1 provides descriptions of the variables in our model, and Table B2 provides descriptive statistics for our student sample. Tables B3 through B5 provide summaries of our regression results. FIGURE B1 Distribution of first -time developmental English enrollment by race/ethnicity SOURCE: Authors’ calculations based on COMIS data. 57 53535660 58 17 1723 20 14 13 11 12 9 11 11 10 8 10 9 7 9 14 6 87 66 6 0 10 20 30 40 50 60 70 80 90 100 Total One-semester acceleration Low-acceleration One level belowTwo levels below Three or more levels below Percent Other African American Asian American White Latino Non -accelerated courses PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 8 TABLE B1 Variable definitions Variable Description Student Outcomes Developmental English Sequence Completer This variable is 1 if a student successfully completed developmental English requirements College composition Student This variable is 1 if a student enrolled in a college composition course College composition Completion This variable is 1 if a student successfully completed a college composition course, conditional on the student enrolling in the course College composition Throughput This variable is 1 if a student successfully completed a college composition course within a given timeframe Total Units Transferable Total number of transferable units earned in a student’s community college career Student demographic attributes Gender Categorical variables for female and unknown sex, with male as the reference category Race/ethnicity Categorical variables for Latino, African American, Asian, other race (includes two or more races, Native American), and unknown race. White is the reference category. Citizenship status Categorical variable for non-citizen (permanent resident, temporary resident, refugee/asylee, F-1 or M -1 student visa, other status) and unknown citizenship. US citizen is the reference category. Traditional College-age Student Categorical variable for a student is age 24 years or younger at first term of enrollment Student academic preparedness proxies Highest level of education at first term Categorical variable for not a high school graduate, adult education, GED or equivalent, graduate from a foreign high school, and unknown education. High school graduate is the reference category Disability status This variable is 1 if a student was ever reported with at least one primary disability (SD01) Limited English Proficiency (LEP) status This variable is 1 if a student was ever enrolled in a course with a TOP Code equal to 493084, 49085, 49086, 493087, or 493100 (English as a Second Language – Writing, Reading, Speaking/Listening, Integrated; Vocational English as a Second Language, respectively) or if the student was identified as ever needing English as a second language services during the matriculation process (SM03) Full -Time Enrollment status This variable is 1 if the student attempted 12 or more units in a term (SX03) Extended Opportunity Programs and Services (EOPS) recipient This variable is 1 if a student every received support from EOPS Enrolled in more than one college This variable is 1 if the student was enrolled in more than one community college during his or her academic career Prior dual enrollment This variable is 1 if the student was previously enrolled as a special admit student (i.e., simultaneously enrolled in K-12) Developmental English student This variable is 1 if a student ever enrolled in at least one developmental English or ESL course Student socioeconomic proxies Board of Governors Enrollment Waiver (BOGW)/Pell recipient This variable is 1 if a student ever received a Board of Governor’s waiver or Pell grant (SF21) NOTES: The CCCCO MIS data element dictionary provides a more detailed description of each variable used in our study (CCCCO u ndated). PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 9 TABLE B2 Student characteristics Students who started two levels below Students in one- semester accelerated courses Total sample Students who started two levels below Students in one- semester accelerated courses Total sample Total 60,591 9,183 69,774 100 100 100 Age at First Term Below 24 56,056 8,613 64,669 93 94 93 Above 24 4,535 570 5,105 7 6 7 Gender Female 31,427 4,621 36,048 52 50 52 Male 28,648 4,473 33,121 47 49 47 Unknown 516 89 605 1 1 1 Race/Ethnicity White 8,333 1,725 10,058 14 19 14 Latino 38,968 4,799 43,767 64 52 63 Asian American 5,771 1,284 7,055 10 14 10 African American 4,450 719 5,169 7 8 7 Native American 323 69 392 1 1 1 Native Hawaiian or Other Pacific Islander 312 100 412 1 1 1 Two or more races 1,853 410 2,263 3 4 3 Unknown race 581 77 658 1 1 1 Citizenship status U.S. citizen 54,094 8,333 62,427 89 91 89 Non-citizen 5,854 827 6,681 10 9 10 Unknown 643 23 666 1 0 1 Highest level of education Not a graduate or no longer enrolled in high school 1,437 158 1,595 2 2 2 Currently enrolled in adult school 244 31 275 0 0 0 High school diploma 52,867 8,093 60,960 87 88 87 Passed GED or received a HS Certificate of Achievement 2,337 357 2,694 4 4 4 California HS Proficiency Certificate 780 145 925 1 2 1 Foreign Secondary School Diploma/Certificate of Graduation 1,190 215 1,405 2 2 2 Unknown 1,736 184 1,920 3 2 3 Previous dual enrollment 0 56,911 8,297 65,208 94 90 93 1 3,680 886 4,566 6 10 7 BOGW or Pell Grant recipient (anytime during his/her college career) 0 12,715 2,570 15,285 21 28 22 1 47,876 6,613 54,489 79 72 78 Ever enrolled in developmental math 0 16,879 2,829 19,708 28 31 28 1 43,712 6,354 50,066 72 69 72 PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 10 Students who started two levels below Students in one- semester accelerated courses Total sample Students who started two levels below Students in one- semester accelerated courses Total sample Full-time student 0 36,025 4,452 40,477 59 48 58 1 24,566 4,731 29,297 41 52 42 EOPS participant 0 56,911 8,297 65,208 94 90 93 1 3,680 886 4,566 6 10 7 Student with limited English proficiency 0 59,741 9,108 68,849 99 99 99 1 850 75 925 1 1 1 Student with disabilities 0 54,590 8,565 63,155 90 93 91 1 6,001 618 6,619 10 7 9 SOURCE : Authors’ calculations based on COMIS data. TABLE B3 Results of probit regression, dependent variable: enrollment in college composition Model 1 Model 2 Model 3 Model 4 Model 5 Coefficient on variable of interest: Accelerated Student 0.927 1.005 1.122 1.147 1.149 (0.065)*** (0.081)*** (0.084)*** (0.074)*** (0.075)*** Student characteristics No Yes Yes Yes Yes Term fixed effects No No Yes Yes Yes College fixed effects No No No Yes Yes Type of sequence No No No No Yes Constant -0.659 -1.765 -0.931 -1.014 -1.017 (0.048)*** (0.122)*** (0.140)*** (0.120)*** (0.124)*** Observations 69,774 69,710 69,710 69,635 69,635 Marginal effect of variable of interest: Accelerated Student 0.305 0.295 0.314 0.302 0.303 (0.017)*** (0.017)*** (0.016)*** (0.019)*** (0.019)*** NOTE: Each column represents a separate regression. Robust standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 11 TABLE B4 Results of probit regression; dependent variable: successf ul completion of college composition Model 1 Model 2 Model 3 Model 4 Model 5 Coefficient on variable of interest: Accelerated Student 0.752 0.788 0.888 0.861 0.858 (0.063)*** (0.079)*** (0.080)*** (0.062)*** (0.063)*** Student characteristics No Yes Yes Yes Yes Term fixed effects No No Yes Yes Yes College fixed effects No No No Yes Yes Type of sequence No No No No Yes Constant -0.847 -1.807 -1.062 -0.938 -0.904 (0.045)*** (0.116)*** (0.138)*** (0.120)*** (0.127)*** Observations 69,774 69,710 69,710 69,624 69,624 Marginal effect of variable of interest: Accelerated Student 0.221 0.207 0.224 0.207 0.207 (0.017)*** (0.017)*** (0.016)*** (0.015)*** (0.015)*** NOTE: Each column represents a separate regression. Robust standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 TABLE B5 Coefficients for OLS regression results; dependent variable: log of total transferable units earned Model 1 Model 2 Model 3 Model 4 Model 5 Coefficient on variable of interest: Accelerated Student 0.216 0.174 0.199 0.2 0.199 (0.084)** (0.081)** (0.081)** (0.031)*** (0.031)*** Student characteristics No Yes Yes Yes Yes Term fixed effects No No Yes Yes Yes College fixed effects No No No Yes Yes Type of sequence No No No No Yes Constant 2.306 1.646 1.686 1.812 1.822 (0.025)*** (0.056)*** (0.070)*** (0.057)*** (0.059)*** Observations 55,288 55,231 55,231 55,231 55,231 NOTE: Each column represents a separate regression. Robust standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 The Public Policy Institute of California is dedicated to informing and improving public policy in California through independent, objective, nonpartisan research. Public Policy Institute of California 500 Washington Street, Suite 600 San Francisco, CA 94111 T: 415.291.4400 F: 415.291.4401 PPIC.ORG P PIC Sacramento Center Senator Office Building 1121 L Street, Suite 801 Sacramento, CA 95814 T: 916.440.1120 F: 916.440.1121" } ["___content":protected]=> string(185) "

Reforming English Pathways at California’s Community Colleges, Technical Appendix

" ["_permalink":protected]=> string(111) "https://www.ppic.org/publication/reforming-english-pathways-at-californias-community-colleges/0218mcr-appendix/" ["_next":protected]=> array(0) { } ["_prev":protected]=> array(0) { } ["_css_class":protected]=> NULL ["id"]=> int(13997) ["ID"]=> int(13997) ["post_author"]=> string(1) "4" ["post_content"]=> string(0) "" ["post_date"]=> string(19) "2018-02-27 12:40:39" ["post_excerpt"]=> string(0) "" ["post_parent"]=> int(13675) ["post_status"]=> string(7) "inherit" ["post_title"]=> string(83) "Reforming English Pathways at California’s Community Colleges, Technical Appendix" ["post_type"]=> string(10) "attachment" ["slug"]=> string(16) "0218mcr-appendix" ["__type":protected]=> NULL ["_wp_attached_file"]=> string(20) "0218mcr-appendix.pdf" ["wpmf_size"]=> string(6) "275078" ["wpmf_filetype"]=> string(3) "pdf" ["wpmf_order"]=> string(1) "0" ["searchwp_content"]=> string(27878) "Reforming English Pathways at California’s Community Colleges Technical Appendices CONTENTS Appendix A. Data and Methods Appendix B. Tables Marisol Cuellar Mejia, Olga Rodriguez, Hans Johnson, and Bonnie Brooks Supported with funding from the Bill and Melinda Gates Foundation, the College Futures Foundation, an d the Sutton Family Fund PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 2 Appendix A. Data and Methods Data This report uses both quantitative and qualitative data to achieve two goals : (1) to describe developmental English course offerings, student enrollment (where available), and outcomes across California’s 114 community colleges, and (2) to evaluate the effectiveness of one of t he most prevalent reforms that colleges are implementing , namely one -semester acceleration. Our quantitative approach utilizes student -level longitudinal data from the California Community College Chancellor’s Office Management Information System (COMIS). The dataset includes students enrolled across the 114 community colleges that comprise the California Community College system, and includes information on student demographics, course -taking behavior, course elements (e.g. , title of course, basic-skills status, minimum/maximum number of credits, etc.), and student outcomes (grades, credits earned, etc.). Table B 1 lists the variables from the COMIS data that we use in our analyses. Data was also collected from an exhaustive scan of college catalogs, websi tes, and other institutional documents. For the purposes of this report, we restrict the universe of developmental English courses to those that are required in order to take college composition. We used C -ID designations to identify college composition courses across colleges. Once we identified the college composition course in each college, we looked for it in the course description section of the college catalog and proceeded backward through the pre -requisites to the lowest level of remedial course wo rk offered by each college that was part of the pathways to college composition. This process allowed us to determine the type of sequence (IRW, separate reading and writing, only writing, or mixed sequence), the number of levels below transfer, and the number of courses required. It also allowed us to identify which colleges are offering one -term accelerated courses and/or co -requisite models. To determine the type of alternative sequence that best characterizes the college, we relied on both our exhaustive scan of college catalogs and actual enrollment data from the MIS data. Classifying courses/sequences in each one of the four types mentioned above—in particular, determining if a course was integrated or not —was challenging. Courses were classif ied as integrated or not based on their name and description. We coded courses as IRW if the course title or description use d the terms “academic reading & writing ,” “critical reading ,” etc. If course titles and descriptions only marginally mentioned readi ng, or if they use a skill s-based description (e.g. vocabulary, reading comprehension, etc.) , we coded them as only writing. To inform our quantitative results, we collected information from interviews with faculty and administrators from California ’s comm unity colleges. We selected interviewees from colleges that (1) had reform pathways with student throughput rates that were significantly better than those of the traditional English pathway and (2) were representative of each alternative English pathway w e studied: the integrated reading and writing pathway and one -term acceleration, with particular attention to the scale of the reform efforts. Out of 24 people we contacted, 13 were interviewed from 11 California community colleges: 12 English faculty and 1 administrator. We spoke with each interviewee for about one hour over the phone. Interviewees were asked a variety of questions pertaining to their background, how students enroll in and learn about alternative English pathways, student experiences in al ternative English pathways, aspects of the alternative pathway course (e.g. , motivation for offering the pathway, course structure , and curriculum), professional development, a s well as funding and scaling up English pathways. Open -ended questions were use d to facilitate the collection of information based on questions that do not restrict the participants’ opinions (Creswell and Plano Clark 2011). PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 3 The data collection and data analysis were carried out simultaneously to avoid the collection of repetitive an d unfocused data (Merriam 1998). Particularly, after each interview was conducted, researchers debriefed, reviewed detailed data notes and audio recordings , and kept notes to capture reflections, emerging themes, and points that needed to be pursued furthe r. This process of review and reflection informed all subsequent interviews. In this manner, data collected from each interview was continuously assessed and informed future interviews until data collection was complete. The data was also organized and cod ed on a secure spreadsheet. This approach was used to come up with a number of patterns and themes. Empirical Strategy for Estimating the Overall Effect of Acceleration in Developmental English on Student Success We focus our analysis on first -time studen ts in the academic year s 2014 –15 and 2015–16, who enrolled in developmental English before or on the fall term of 2016. We restrict our sample to (1) students with valid social security numbers and birthdays, (2) first -time students age 15 –64 at time of entry into the California community college system who (3) took an English or math course in their community college career, and (4) were only enrolled in either the accelerated course or a multilevel developmental English sequence. We exclude students who hold bachelor’s or associate degrees at time of entry , students enrolled in only summer or winter terms, and dual -enrollment students. After implementing these restrictions, our sample of interest include d 69,774 first-time students in the CCC system (15 % of which were enrolled in an accelerated course). To test the hypothesis that students enrolled in accelerated courses are more likely to enroll and succeed in college composition than students that went through a multilevel sequence, we use probit and ordinary least square fixed- effects specifications. Students who took the accelerated courses are selected into the treatment group. Our control group is any student who started the developmental sequence two levels below transfer. While there are a few differences between treatment and control groups, student characteristics are similar on the whole, which allows us to fairly compare these groups in our analyses. Our study focuses on four key outcomes: completion of developmental English, enrollment in college composition, successful completion of college composition (throughput), and total credits earned. Our regression models take the following form , where i indexes students and g colleges: ���� ��������= ��������+ ���� PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 4 course prior to college composition. In other words, we exclude from our treatment group any student who took one or more courses as part of the multilevel sequence before enrolling in the accelerated pathway. However, this model may generate biased estimates of ���� . The most notable arises due to omitted variable bias. Even when we control for a number of student characteristics, it is very difficult to measure other characteristics such as motivation or student ability upon college entry. Because these measures are likely correlated with enrollment in developmental education and success in college, even when students have similar observed characteristics, the inability to control for motivation and pr ior ability could lead to biased estimates. In addition, selection bias may arise due to differential enrollment patterns in developmental education. For instance, if the most motivated students are more likely to enroll in required developmental courses w hile the least motivated are more likely to delay or never enroll in developmental coursework, we can observe upward bias. We also run separate models that include interaction terms to evaluate the effect of acceleration by student attributes (race, gender , low-income status) and by college. The college- interacted model is especially noteworthy, as it assures our results are not solely driven by a single college. Comparing Our Estimates to Those in the Existing Literature Our study of accelerated developmental English in California’s community colleges uses data from over 700,000 students who first enrolled in any community college during the 2014– 15 and 2015–16 academic years. Using multivariate regressions that control fo r a variety of student, college, and course characteristics, we found that students who take a one -term accelerated developmental English course are substantially more likely to go on and complete a college- level English course than students who take the t wo-level sequence. These results hold by gender and by ethnic group; however, we find that equity gaps remain. The findings in our study are consistent with the evidence found by prior studies on one -term acceleration in the state of California and elsewhere in the nation. One of the earliest pieces of descriptive evidence on one- term acceleration in California comes from Chabot College. Looking at cohorts going back as far as 2000, Hern (2011) found that students enrolling in the open- access one- term accelerated pathway had completed college English at significantly higher rates within three years than students who started in a non- accelerated two-term pathway (54% vs. 28% for the fall 2008 cohort). Furthermore, students from all racial/ethnic groups exper ienced gains under the accelerated pathway; however, the gaps between racial/ethnic groups remained and in some instances widened. The overall findings of this research were subsequently corroborated in a quasi -experimental study of acceleration at Chabot by Edgecombe and colleagues (2014). The analysis drew from the data of 3,853 accelerated English students and 4,757 non- accelerated English students that enrolled for the first time at Chabot College between 1999 and 2010. The study employed propensity sco re matching (PSM) and regression analyses and controlled for student demographic and academic background to examine academic outcomes up to five years after entry. Overall, the research found that participation in the one -term acceleration course was posit ively associated with college English completion, credit accumulation, grade point average, certificate and degree attainment and transfer; but in some instances the gains for accelerated students diminished over time. Specifically, in the PSM model a 25 percentage point gain in transfer English completion within one year declined to 22 percentage points after five years. Furthermore, the PSM model found that the gains in transfer -ready status and graduation declined or became non -significant after five yea rs. However, the advantage students who took accelerated courses experienced in terms of grades and credit accrual was maintained over this timeframe. This study did not explore whether the impacts of acceleration varied for different racial/ethnic or gend er groups. PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 5 Our findings are also in line with Hayward and Willett’s (2014) study that examined the impact of accelerated developmental English pathways at ten community colleges in California. These pathways, implemented using the curricular and pedagogic al principles developed by the California Acceleration Project (CAP), provide an alternative to the traditional multilevel sequence composed of separate reading and writing courses. The accelerated pathway is intended to reduce the time students spend in developmental English and make the skills students learn more relevant to the work they will be expected to do in college -level English. The analysis drew from the data of 1,836 accelerated English students and 22,354 traditional English students enrolled a t 10 colleges during the 2011 –12 academic year and tracked outcomes for two years. The study employed a multivariate logistic regression with student background controls and found that the odds of completing college -level English for accelerated English st udents were 1.5 times higher than for students in the traditional English sequence. Importantly, they found that the odds varied significantly by college —colleges that granted open access to a one- term accelerated English approach (high acceleration) achieved odds of 2.3, compared to odds of 1.2 at colleges that required students to complete a pre -requisite course (low acceleration). Additionally, the study found that the accelerated approach improved outcomes for minority students, but gaps remained. The r esearch provides among the most promising evidence of developmental English reform available in the state of California and this has led to a surge in colleges adopting this reform. Research by Hodara and Jaggars (2014) also sheds light on the impact of ac celeration in the form of offering a shorter (one -term) versus a longer (two -term) developmental writing sequence. This research drew from the data of 3,932 accelerated English students and 7,498 non- accelerated English students that enrolled for the first time at a City University of New York community college between 2001 and 2007. The study employed propensity score matching and controlled for student background and cohort fixed effects to examine academic outcomes within three and five years after entry . The researchers found that students who started in a shorter, one -term writing sequence were 9.7 percentage points more likely to enroll in college English than their peers who started in a two - term sequence; students in the shorter sequence were also 6 percentage points more likely to complete college English within three years. However, this overall completion increase comes at a cost, when condition on enrollment in the college English course, accelerated students were found to pass the college English course at a lower rate (2.5 percentage points lower). Still, when examining longer -term outcomes, accelerated students outperformed their nonaccelerated peers. Namely, students who enrolled in the shorter writing sequence earned about two more credits ove r three years and earned a degree at a higher rate after five years (2.2 percentage points higher) than their peers who started in a longer writing sequence. While this study did not explore differences by race/ethnic or gender groups, it does shed light on the promise of developmental education in a very large and diverse urban community college system (over 70% of students are Latino and African American). While this evidence base suggests that accelerating student progression through developmental English by providing a one -term developmental English pathway has both positive early impacts and potential for longer - term positive impact, it is less clear whether the strategies in fact help close equity gaps for diverse groups of students. Also, while the ov erall evidence on one-term acceleration is promising, it is important to acknowledge that a newer approach to acceleration known as co- requisite remediation (discussed in detail in the main report) has also gained much momentum due to positive early findings (Cho et al. 2012; Coleman 2015; Denley 2016; Jaggars, et al. 2015; Jenkins et al. 2010). Overall, the mounting evidence on the benefits of acceleration has spurred adoption of reforms at colleges across California. Our review of the research suggests mo re work is needed to explore whether the impact of acceleration strategies persists beyond the first year and whether they help address equity gaps. PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 6 Caveats While our analysis employs rigorous statistical analysis and includes a substantial number of con trols, there are considerations to take into account with regard to the data we use and our methods. Our data does not contain information pertaining to student assessment scores, placement recommendation s, or high school GPA, all of which would more accurately control for student s’ academic record prior to enrolling in developmental English. We use students ’ first course in the developmental English sequence as a proxy variable. We track student outcomes for anywhere between 1 and 10 terms. While the sampl e sizes we do include in our study are sufficient to draw conclusions from our regression analyses, our analysis w ould have benefited from having larger sample sizes of students enrolled in accelerated pathways. One reason for having smaller -than- desired sample sizes is the recent implementation of these accelerated pathways. Our analysis focuses on enrollment in accelerated pathways in relation to student outcomes. However, other types of reforms were implemented within our timeframe of interest that may also have contribute d to our results, most notably assessment and placement reforms such as multiple measures and adjustments to the cut scores used to place students. Including college fixed effects in our model aims to account for these other reforms implemented at the college level, outside of alternative math pathways. PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 7 Appendix B. Tables In each of our regression models, we compare developmental education students in accelerated pathways to students who are in non- accelerated pathways that begin tw o levels below college composition. We develop three separate sets of models for the following dependent variables: (1) enrollment in college composition; (2) successful completion of college composition (also known as throughput); and (3) number of trans fer units completed. Table B1 provides descriptions of the variables in our model, and Table B2 provides descriptive statistics for our student sample. Tables B3 through B5 provide summaries of our regression results. FIGURE B1 Distribution of first -time developmental English enrollment by race/ethnicity SOURCE: Authors’ calculations based on COMIS data. 57 53535660 58 17 1723 20 14 13 11 12 9 11 11 10 8 10 9 7 9 14 6 87 66 6 0 10 20 30 40 50 60 70 80 90 100 Total One-semester acceleration Low-acceleration One level belowTwo levels below Three or more levels below Percent Other African American Asian American White Latino Non -accelerated courses PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 8 TABLE B1 Variable definitions Variable Description Student Outcomes Developmental English Sequence Completer This variable is 1 if a student successfully completed developmental English requirements College composition Student This variable is 1 if a student enrolled in a college composition course College composition Completion This variable is 1 if a student successfully completed a college composition course, conditional on the student enrolling in the course College composition Throughput This variable is 1 if a student successfully completed a college composition course within a given timeframe Total Units Transferable Total number of transferable units earned in a student’s community college career Student demographic attributes Gender Categorical variables for female and unknown sex, with male as the reference category Race/ethnicity Categorical variables for Latino, African American, Asian, other race (includes two or more races, Native American), and unknown race. White is the reference category. Citizenship status Categorical variable for non-citizen (permanent resident, temporary resident, refugee/asylee, F-1 or M -1 student visa, other status) and unknown citizenship. US citizen is the reference category. Traditional College-age Student Categorical variable for a student is age 24 years or younger at first term of enrollment Student academic preparedness proxies Highest level of education at first term Categorical variable for not a high school graduate, adult education, GED or equivalent, graduate from a foreign high school, and unknown education. High school graduate is the reference category Disability status This variable is 1 if a student was ever reported with at least one primary disability (SD01) Limited English Proficiency (LEP) status This variable is 1 if a student was ever enrolled in a course with a TOP Code equal to 493084, 49085, 49086, 493087, or 493100 (English as a Second Language – Writing, Reading, Speaking/Listening, Integrated; Vocational English as a Second Language, respectively) or if the student was identified as ever needing English as a second language services during the matriculation process (SM03) Full -Time Enrollment status This variable is 1 if the student attempted 12 or more units in a term (SX03) Extended Opportunity Programs and Services (EOPS) recipient This variable is 1 if a student every received support from EOPS Enrolled in more than one college This variable is 1 if the student was enrolled in more than one community college during his or her academic career Prior dual enrollment This variable is 1 if the student was previously enrolled as a special admit student (i.e., simultaneously enrolled in K-12) Developmental English student This variable is 1 if a student ever enrolled in at least one developmental English or ESL course Student socioeconomic proxies Board of Governors Enrollment Waiver (BOGW)/Pell recipient This variable is 1 if a student ever received a Board of Governor’s waiver or Pell grant (SF21) NOTES: The CCCCO MIS data element dictionary provides a more detailed description of each variable used in our study (CCCCO u ndated). PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 9 TABLE B2 Student characteristics Students who started two levels below Students in one- semester accelerated courses Total sample Students who started two levels below Students in one- semester accelerated courses Total sample Total 60,591 9,183 69,774 100 100 100 Age at First Term Below 24 56,056 8,613 64,669 93 94 93 Above 24 4,535 570 5,105 7 6 7 Gender Female 31,427 4,621 36,048 52 50 52 Male 28,648 4,473 33,121 47 49 47 Unknown 516 89 605 1 1 1 Race/Ethnicity White 8,333 1,725 10,058 14 19 14 Latino 38,968 4,799 43,767 64 52 63 Asian American 5,771 1,284 7,055 10 14 10 African American 4,450 719 5,169 7 8 7 Native American 323 69 392 1 1 1 Native Hawaiian or Other Pacific Islander 312 100 412 1 1 1 Two or more races 1,853 410 2,263 3 4 3 Unknown race 581 77 658 1 1 1 Citizenship status U.S. citizen 54,094 8,333 62,427 89 91 89 Non-citizen 5,854 827 6,681 10 9 10 Unknown 643 23 666 1 0 1 Highest level of education Not a graduate or no longer enrolled in high school 1,437 158 1,595 2 2 2 Currently enrolled in adult school 244 31 275 0 0 0 High school diploma 52,867 8,093 60,960 87 88 87 Passed GED or received a HS Certificate of Achievement 2,337 357 2,694 4 4 4 California HS Proficiency Certificate 780 145 925 1 2 1 Foreign Secondary School Diploma/Certificate of Graduation 1,190 215 1,405 2 2 2 Unknown 1,736 184 1,920 3 2 3 Previous dual enrollment 0 56,911 8,297 65,208 94 90 93 1 3,680 886 4,566 6 10 7 BOGW or Pell Grant recipient (anytime during his/her college career) 0 12,715 2,570 15,285 21 28 22 1 47,876 6,613 54,489 79 72 78 Ever enrolled in developmental math 0 16,879 2,829 19,708 28 31 28 1 43,712 6,354 50,066 72 69 72 PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 10 Students who started two levels below Students in one- semester accelerated courses Total sample Students who started two levels below Students in one- semester accelerated courses Total sample Full-time student 0 36,025 4,452 40,477 59 48 58 1 24,566 4,731 29,297 41 52 42 EOPS participant 0 56,911 8,297 65,208 94 90 93 1 3,680 886 4,566 6 10 7 Student with limited English proficiency 0 59,741 9,108 68,849 99 99 99 1 850 75 925 1 1 1 Student with disabilities 0 54,590 8,565 63,155 90 93 91 1 6,001 618 6,619 10 7 9 SOURCE : Authors’ calculations based on COMIS data. TABLE B3 Results of probit regression, dependent variable: enrollment in college composition Model 1 Model 2 Model 3 Model 4 Model 5 Coefficient on variable of interest: Accelerated Student 0.927 1.005 1.122 1.147 1.149 (0.065)*** (0.081)*** (0.084)*** (0.074)*** (0.075)*** Student characteristics No Yes Yes Yes Yes Term fixed effects No No Yes Yes Yes College fixed effects No No No Yes Yes Type of sequence No No No No Yes Constant -0.659 -1.765 -0.931 -1.014 -1.017 (0.048)*** (0.122)*** (0.140)*** (0.120)*** (0.124)*** Observations 69,774 69,710 69,710 69,635 69,635 Marginal effect of variable of interest: Accelerated Student 0.305 0.295 0.314 0.302 0.303 (0.017)*** (0.017)*** (0.016)*** (0.019)*** (0.019)*** NOTE: Each column represents a separate regression. Robust standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 PPIC.ORG T echnical Appendices Reforming English Pathways at California’s Community Colleges 11 TABLE B4 Results of probit regression; dependent variable: successf ul completion of college composition Model 1 Model 2 Model 3 Model 4 Model 5 Coefficient on variable of interest: Accelerated Student 0.752 0.788 0.888 0.861 0.858 (0.063)*** (0.079)*** (0.080)*** (0.062)*** (0.063)*** Student characteristics No Yes Yes Yes Yes Term fixed effects No No Yes Yes Yes College fixed effects No No No Yes Yes Type of sequence No No No No Yes Constant -0.847 -1.807 -1.062 -0.938 -0.904 (0.045)*** (0.116)*** (0.138)*** (0.120)*** (0.127)*** Observations 69,774 69,710 69,710 69,624 69,624 Marginal effect of variable of interest: Accelerated Student 0.221 0.207 0.224 0.207 0.207 (0.017)*** (0.017)*** (0.016)*** (0.015)*** (0.015)*** NOTE: Each column represents a separate regression. Robust standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 TABLE B5 Coefficients for OLS regression results; dependent variable: log of total transferable units earned Model 1 Model 2 Model 3 Model 4 Model 5 Coefficient on variable of interest: Accelerated Student 0.216 0.174 0.199 0.2 0.199 (0.084)** (0.081)** (0.081)** (0.031)*** (0.031)*** Student characteristics No Yes Yes Yes Yes Term fixed effects No No Yes Yes Yes College fixed effects No No No Yes Yes Type of sequence No No No No Yes Constant 2.306 1.646 1.686 1.812 1.822 (0.025)*** (0.056)*** (0.070)*** (0.057)*** (0.059)*** Observations 55,288 55,231 55,231 55,231 55,231 NOTE: Each column represents a separate regression. Robust standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1 The Public Policy Institute of California is dedicated to informing and improving public policy in California through independent, objective, nonpartisan research. Public Policy Institute of California 500 Washington Street, Suite 600 San Francisco, CA 94111 T: 415.291.4400 F: 415.291.4401 PPIC.ORG P PIC Sacramento Center Senator Office Building 1121 L Street, Suite 801 Sacramento, CA 95814 T: 916.440.1120 F: 916.440.1121" ["post_date_gmt"]=> string(19) "2018-02-27 20:40:39" ["comment_status"]=> string(4) "open" ["ping_status"]=> string(6) "closed" ["post_password"]=> string(0) "" ["post_name"]=> string(16) "0218mcr-appendix" ["to_ping"]=> string(0) "" ["pinged"]=> string(0) "" ["post_modified"]=> string(19) "2018-02-27 12:52:03" ["post_modified_gmt"]=> string(19) "2018-02-27 20:52:03" ["post_content_filtered"]=> string(0) "" ["guid"]=> string(59) "http://www.ppic.org/wp-content/uploads/0218mcr-appendix.pdf" ["menu_order"]=> int(0) ["post_mime_type"]=> string(15) "application/pdf" ["comment_count"]=> string(1) "0" ["filter"]=> string(3) "raw" ["status"]=> string(7) "inherit" ["attachment_authors"]=> bool(false) }