How Effective for Learning are the Ed Tech Apps we Use?

As the costs for devices and applications decrease, technology use is increasing In classrooms across the country. Growth is particularly high in web-based devices. A recent University of Phoenix College of Education survey showed that about 58% of teachers use educational apps and 41% use social media. As tablets, Chromebooks, and other handheld devices are employed in classrooms, instruction relies more and more on apps instead of on traditional computer-based programs.

With so much learning dependent on app-based technology, educators, administrators, and communities need to know if they are using the best apps for each purpose and for each curriculum area. Cost and usage are only part of the picture. Even more, they need to know the impact of these apps — if students are learning because of them.

Impact tracks the link between student usage on a particular app and how that may or may not influence their performance on assessments.

More than $3 billion was spent on ed tech products last year and more than 2,500 programs are on the market. Educators need to know which products are likely to have a positive impact on learning for their students. To figure that out, they can look at assessment data for evidence of correlation between apps and test score improvement. With this knowledge, administrators can leverage their data to make smart decisions about using the apps that will best improve learning.

Dr. Ryan S. Baker, Director of the Penn Center for Learning Analytics and Associate Professor, Graduate School of Education, studied 1.4M hours of usage data from 392,603 students in 48 districts to put together a comprehensive report that reveals surprising findings on the range of cost per license, the degree ed tech apps are being used, and the level of app effectiveness for math, science, and ELA.

His groundbreaking study produced insights into the national trends of cost, usage and impact of apps across a broad cross-section of school districts in the U.S. His research also analyzed the contexts where apps are producing positive impact. In addition, he uncovered just how many apps are purchased by districts for use in classrooms but are not widely used. (Spoiler alert: The median percent of unused licenses is 70.1%)

Investment, Engagement, Impact

National data is meaningful but districts also need data that shows if they are making the best decisions on apps for their classrooms. Analyzing their data can tell them which apps work best in which classrooms, which have the greatest impact on learning, and the conditions in which specific apps are effective. The key is in evaluating data about the cost, user perception and the effectiveness of apps the district has employed. Together the data will provide answers about the districts’ technology investment, engagement, and impact.

While there are data mining programs that report cost and usage, the key in understanding the value of a district’s technology decisions is to use data that shows the impact on learning. To break down the analysis needed, districts should connect program usage and student achievement.

We know that if an app is seldom used, it can’t be effective. What we don’t know is impact. If an app for literacy instruction offers an individualized approach and provides performance data within the app, that data should be mapped to the results of reading proficiency assessments to judge its value. If a math program with a common core aligned curriculum allows teachers to assign content for students to practice online and track their progress toward mastery, that data should be mapped to the results of math assessments to gauge its effectiveness.

Some districts have implemented BrightBytes’ Learning Outcomes Modules, which captures data using a district’s existing web proxy and employs advanced research-driven analytics to correlate usage with student achievement data. It takes millions of data points from digital device usage and triangulates it with district investment data and student achievement data.

With this data, educators can know which programs have the most impact and can develop plans to replicate, scale, or adjust resources accordingly.

The future of employing apps in classrooms is based on knowing what makes a difference for student learning by assessing the impact on student scores and gauging how much each app is used in order to gain insight into the real costs. Having the right data leads to informed decision-making and future student success.