Are California’s Schools Ready for Online Testing and Learning?
SummaryIn addition to the Common Core State Standards (CCSS), California is implementing a new, online assessment system: the California Assessment of Student Performance and Progress (CAASPP). Field tests were conducted last spring and the system is being rolled out this year, amid concerns about whether schools are technologically prepared. Using survey data from the California Educational Technology Professionals Association (CETPA), this report examines school districts’ technology infrastructure and assesses their readiness for online testing. Three findings emerge. First, school districts express confidence in the quantity and quality of their hardware and network capabilities but remain concerned about software and training of instructional and IT staff. Second, there is sizable variation in readiness across districts, linked mainly to student enrollment and district expenditure levels. Third, a clear majority of the state’s onetime CCSS Implementation Fund is going into non-technology spending such as instructional materials and teacher training. Regardless of their current readiness, districts will need targeted and ongoing support to upgrade and maintain their technology infrastructure. In the longer term, virtually all schools will need to upgrade their technology infrastructure in order to adopt and benefit from digital learning.
ASSESSING TECHNOLOGICAL READINESS
In spring 2014, more than 3.1 million California students took part in a field test of the Smarter Balanced assessments, part of the CAASPP system aligned with the Common Core standards. As the state implements testing in 2015, there are still concerns about whether schools and districts have enough capacity to deliver the tests, which are administered online. And as California moves toward developing digital learning, schools and districts will need to upgrade their technological infrastructures. To assess both levels of preparedness, we analyze the best available data on technology readiness in California schools.
Comprehensive information on technology readiness across California schools is limited.1 This report relies on the 2014 CAASPP Field Survey conducted by the California Educational Technologist Professionals Association (CETPA). The original survey was conducted in spring 2014 and a follow-up was done in fall 2014. About a third (362, or 35%) of California’s districts responded to the survey; large and urban districts are overrepresented but other factors (such as the share of English Learners and the share of students eligible for free and reduced-price lunch) appear to be proportionately represented in the survey. The CEPTA survey is not a perfect data source—like all surveys with high rates of non-response, it must be interpreted carefully because its respondents are not a random sample. On the plus side, the survey was designed in consultation with technology officers at county offices of education and sent to district technology directors and staff, so it is reasonable to assume that the questions were clearly understood and answered. For more information on the survey and associated respondent analysis, please refer to Technical Appendices A and B.
DISTRICT READINESS FOR TESTING
Figure 1 summarizes districts’ technology readiness in four major areas.2
Hardware: desktops or devices; specifications or quality of hardware
Network: bandwidth; network reliability
Software: installation and reliability of secure browser; distribution of IDs and quick log-ins
Staffing: technical support; training for instructional staff, support staff, and assessment technicians
Figure 1. Software and staffing levels are insufficient in many districts
Districts expressed confidence in their stock of physical capital such as hardware and network but were concerned about software and even more so about training instructional and technical staff. 3 More than 70 percent of survey respondents reported a sufficient amount of hardware and network for all students to take the test.4 However, only half were confident about their ability to handle software-related issues such as the installation of secure browsers and the distribution of IDs and quick log-ins for students. Some districts experienced difficulty installing the correct software on netbooks (e.g., Chromes or iPads); others were worried about the reliability of their secure browsers; and some districts expressed concerned about the time it took for students to log in.
In addition, a clear majority of districts reported problems with staffing. For instance, 68 percent of schools do not have sufficient staff to provide technical support or do not provide sufficient training for instructional staff, support staff, and assessment technicians. Anecdotal information suggests that a lack of training and professional development has been a key barrier to major deployment of technology. For instance, one of the hard lessons from LA Unified School District’s ambitious attempt to provide all students with digital computing devices is the importance of frequent and high-quality training for teachers and technical staff.5
Among districts with sufficient hardware, the vast majority (86%) have reliable networks (and vice versa: districts with sufficient networks tend to have sufficient hardware). However, even districts with sufficient resources in the other two areas tend to lack software and staffing. Only 57 percent have sufficient software, and even fewer—38 percent—have sufficient staffing (Table 1). This suggests that districts need technical assistance with software and particularly with staffing, regardless of their readiness in other areas.
Table 1. Staffing and software tend to be problems even for districts with sufficient resources in other areas
ARE DISTRICTS INVESTING IN TECHNOLOGY UPGRADES?
The CETPA survey results suggest that districts need to invest in key components of technology infrastructure, particularly staffing. However, per pupil spending on technology is at historically low levels and it is not clear whether this trend will change without significant assistance from the state.6 In the 2013–14 school year, the legislature appropriated $1.25 billion in onetime CCSS implementation funding to help schools pay for teacher training, instructional materials, and technology upgrades. But most districts spent less than half of their funding on technology-related purchases; they opted to allocate a clear majority of the funds for teacher training and instructional materials (Figure 2).7
Figure 2. Only a few schools spent most or all of their CCSS implementation funding on technology
If maintaining and upgrading district technology infrastructure is a priority, the state could provide targeted funds dedicated to technology. In addition to financial investment, the state could also provide technical assistance to lagging districts, particularly related to software and staff training.
Districts Need Ongoing Support
A one time appropriation is unlikely to cover the full cost of technology upgrades, which normally take place over a few years. In fact, significant shares of district IT budgets are dedicated to ongoing maintenance and replacement costs. Policymakers need to take the ongoing nature of technology spending into consideration as they consider how to support local technology infrastructure.
FACTORS AFFECTING DISTRICT READINESS
There is wide variation in the technology readiness of California districts.8 Student enrollment and district spending are closely linked to district readiness. Other factors, such as student composition, student performance, district location, and neighborhood do not contribute to disparities in readiness (Table 2).9 This seems surprising, given anecdotal evidence that schools with high concentrations of disadvantaged and/or low-performing students tend to have more difficulty adopting new technology.10
Table 2. Disparities in technological readiness do not seem to be linked to most district characteristics
District size and spending levels drive technology readiness even after we control for district location, student composition, student performance, and county fixed effects. Figure 3 shows the effect of district size and spending levels on technological readiness in the four major areas.
Figure 3. The impact of student enrollment and district spending levels on district readiness varies across technological areas
For example, a thousand-student increase in enrollment is related to a slight but significant decrease in software readiness, but does not appear to be statistically related to hardware, network, and staffing. Per pupil spending is strongly correlated with software and staffing. For instance, a thousand-dollar increase in spending translates into a significant jump in software readiness, enough to make districts with the least sufficient infrastructure technologically ready. The magnitude is markedly larger in staffing, perhaps because 68 percent of districts had trouble in this area, and money thus plays a more critical role. On the other hand, the amount of spending does not seem to matter when it comes to hardware or networks. This is likely because both are sufficient in more than 70 percent of districts.
PREPARING FOR DIGITAL LEARNING
In 2013 the Obama administration introduced an ambitious digital learning initiative called ConnectED. The initiative is designed to help schools upgrade their technology and train teachers to use video-conferencing, virtual field trips, personalized audio-visual learning, and other digital learning tools. The recommended minimum bandwidth for digital learning is 100 megabits per second (Mbps)—with a target of 1gigabit per second (Gbps). In California, 39 percent of schools and 44 percent of districts are connected at speeds less than 100 Mbps.11
Dis-economy of Scale
Based on an assessment of bandwidth data from K–12 High Speed Network, we find that network upgrades are particularly challenging for small and large districts. It makes intuitive sense that upgrades for small numbers of students involve high per student costs. Initially as student enrollment grows, it becomes more cost-effective to upgrade to a faster network; however, after a certain point (around 28,000 students), increases in student enrollment make upgrading less cost-effective (see technical appendix Figure E1). This dis-economy of scale makes it more costly to upgrade bandwidth in particularly small and large districts.12
Digital Divide between Urban and Rural Districts
We also find that bandwidth sufficiency is linked to geographic location. Eighteen percent of urban districts lack sufficient bandwidth (i.e., a minimum of 100 Mbps) for digital learning and the number is 48 percentage points higher in rural districts (Figure 4). Because of the relatively low density and greater geographical distances in rural areas, affordable broadband services may be more difficult for rural districts to obtain.13
Figure 4. Rural schools and districts are being left behind in the digital race
Programs That Can Help Close Gaps
The federal E-Rate program provides discounted Internet services to schools based on the composition of their student bodies and their geographic location. At the state level, the California Teleconnect Fund also has the potential to transform rural districts. The state needs to streamline these programs and ensure that all eligible rural schools receive significant discounts. The state also needs to provide technical assistance to rural districts, which are more prone to staffing challenges.
Our analysis of survey data produced some important insights about technological readiness in California’s schools. For example, we know there are stark differences in technological infrastructure in schools and districts across the state. But policymakers could understand and pinpoint IT needs more accurately if they collected better information. The state currently collects and publishes data on number of computers (per student) and number and percentage of classrooms connected to the Internet, which bear little relationship to schools’ technology readiness (see Table D1 in the technical appendices). More refined data on IT infrastructure, from hardware (e.g., quantity and quality of computers/devices, student-device ratio), network (e.g., bandwidth, reliability, firewall), to staffing (e.g., student/technology staff ratio, frequency and quality of training for technology and instructional staff) will provide a more complete picture on schools’ IT infrastructure. With these data, the state could identify specific needs and provide targeted technical assistance to lagging districts.
Addressing schools’ technological gaps and needs seems to be a priority at both the state and federal levels. At the state level, Governor Brown included another $27 million in his 2014–15 budget to support Broadband Infrastructure Improvement Grants. At the federal level, the newly reformed E-Rate program, with its funding now doubled to $2.4 billion, provides significant discounts on telecommunication and Internet access to schools and districts. While these programs have the potential to level the playing field, it is too early to gauge their effectiveness.
As they pursue these efforts, policymakers and districts responsible for budgetary decisions should recognize the need for ongoing technological upgrades and maintenance, which often involve multiyear investments. Targeted support for IT staffing is also especially important. Finally, since school enrollment and expenditure levels are the main drivers of technology readiness, the state should pay special attention to large and historically low-spending districts. Targeted and coordinated efforts to upgrade and maintain the technology infrastructure in California’s public schools will give all students a chance to benefit from digital learning.
I am grateful to Sarah Bohn for her unfailing support and exceptionally high standards. I am greatly indebted to Andrea Bennett for giving me access to the 2014 CAASPP survey data and helping me make sense of it. I thank John Wells, Steven Ross, Paul Warren, and Lynette Ubois for excellent comments, and Mary Severance for excellent editorial support. I benefited tremendously from conversations with Teri Sanders and Robert Gravina. Maggie Weston, a beloved colleague, provided support and insight until her passing in July 2014. I am extraordinarily lucky to have worked with Maggie—without her encouragement, this report would never have been finished. Any errors are my own.
- The state collects data on the computers less than 48 months old, the number of students per each computer that is less than 48 months old, and the number of classrooms with high-speed Internet. School and district bandwidth data are also available from K–12 High Speed Network. (K12 HSN), a state program funded by the California Department of Education. K12 HSN administers K?12’s participation in the California Research and Education Network (CalREN), which offers network and Internet services to K?12, UC, CSU, the community colleges, and participating private universities.
- For a detailed discussion of each component, please refer to Technical Appendix C.
- Throughout this report, we use (unweighted) responses from the 2014 CAASPP Survey. One could weight the survey data to (partially) address the non-randomness in survey response. We do not pursue this method for two reasons. First, as shown in Technical Appendix B, respondents differ from nonrespondents in many ways, which makes the weighting too complicated. Second and more important, we are concerned that respondents may be different in unobservable ways, and that weighting based on observables might therefore exacerbate the bias. We thank one of our reviewers for pointing this out.
- The survey did not define “sufficient” explicitly. Instead, it asked district staff about their perceptions. Districts have a window of several days over which they can distribute their testing—they do not have to test all students at the same time. This may explain why districts with slower networks feel they have sufficient bandwidth.
- Benjamin Herold, “Hard Lessons Learned in Ambitious L.A. iPad Initiative,” Edweek, September 2014; “The Inside Story on LA Schools’ iPad Rollout: ‘A Colossal Disaster,’” blogpost, Hechinger Report, September 30, 2013.
- Ideally, we would like to know the actual spending on technology infrastructure; however, such data do not exist. On average, less than 10 percent of district budget is dedicated to capital outlay. Since capital outlay includes more than just technology infrastructure, we suspect the actual spending on technology is even lower.
- Schools may prioritize teacher training and instructional materials because the implementation funding is not sufficient for fully implementing the CCSS. Without adequate school budget data, it is difficult to see how much money goes into technology and why schools are underinvesting in technology. We thank one of our reviewers for pointing this out.
- There is also variation across schools within individual districts. For instance, some schools are more capable of administering online testing than other schools in the same district. We thank one of our reviewers for pointing out the importance of variation within districts, but we do not have appropriate data to explore this issue.
- This does not rule out the possibility that these factors may matter at the individual school level. Also, the finding that neighborhood factors such as median household income or average property value appear to matter less than district spending and district size should be interpreted with caution: a closer look at this question requires a more granular dataset that goes beyond the reach of this study.
- Anecdotal sources suggest that students from disadvantaged families may have more difficulty adapting to online testing, resulting in a technology gap along the line of family income. To explore this, I tried alternative specifications that include neighborhood factors such as median household income or average property value, which are not significant at any conventional level. There are two possible explanations. First, median household income and average property value may be noisy proxies for family income and parental involvement, which directly affect student familiarity with testing tools such as laptops and tablets. Second, answering this question requires a more granular database such as data at individual school or student level. See technical appendix Table D1.
- The discrepancy between school and district bandwidth could be explained by the fact that some schools are not connected via district networks.
- This is based on a linear regression of bandwidth on a variety of characteristics, including district enrollment, district urbanicity; median household income; district spending, student composition, student need, student performance, and county fixed effects. Standard errors are adjusted for clustering at county level. Please refer to Table E1 in the technical appendices for a detailed discussion of model specification. The inverse u-shaped curve is identified as follows: first run a quadratic regression and identify the point at which the u-shape maxes out; then run a linear regression up to the tipping point (27,824) and another from that point onward. The second line is negative and significant at 1 percent.
- Interestingly, there is a positive relationship between percent Hispanic students and district connectivity. The effects are very small compared to district enrollment and urbanicity, however.