The Value of Rapid Cycle Evaluations (RCE) with IMPACT™
More and more, people are searching for evidence to inform their decisions regarding effective edtech. How can you be confident that a product benefits your students? LearnPlatform allows you to conduct accessible and timely research in your context through the use of Rapid Cycle Evaluations (RCE) with IMPACT™. Through this process, you can determine how a product is being used (Usage Analysis), understand how use of a product impacts students (Outcome Analysis), and/or uncover potential cost savings (Cost Analysis) as it relates to your organization, teachers, and students. This helps you make necessary adjustments to the implementation of edtech products within your organization to improve results.
What is the RCE with IMPACT™ process?
RCE with IMPACT™ is a formative decision-making process that allows you to examine real-world data and make decisions about what works, under what conditions, and for whom. Specifically, LearnPlatform’s RCE process involves:
- Defining a research goal
- Identifying, collecting, and preparing data for analysis
- Analyzing data and reporting results
- Looking for success points (i.e., subgroups where the product has a positive impact)
- Deciding on implementation changes based on a review of success points
Users can review the findings with their teams, make implementation improvements, and repeat the RCE cycle to continuously improve. This allows for rapid reporting and decision-making to ensure high levels of success.
What is the RCE reporting and analysis tool, IMPACT™?
Developed by LearnPlatform, the RCE reporting and analysis tool, IMPACT™ (Integrated Metrics Producing Analytics on Classroom Technologies), is an evidence-based, analytical methodology that integrates data from multiple sources (e.g., usage, achievement, demographics, cost) to generate insights into the implementation and impact of educational technology interventions. IMPACT™ produces multiple RCE reports and dashboards, maximizing insights by analyzing data on product effectiveness and teacher feedback. This state-of-the-art methodology employs sophisticated analytics that help schools and districts better understand how edtech is being used in their organizations and which products contribute to meaningful education outcomes (e.g., engagement, achievement, 21st century skills). This informs critical instructional, operational, and financial decisions, allowing administrators to identify and implement the most effective educational interventions for their classrooms.
What differentiates RCE with IMPACT™?
RCE with IMPACT™ is driven by a scientific methodology designed to deliver practical, on-demand insights that inform instructional, operational, and financial decisions. The research-backed methodology includes a proprietary grading rubric, scoring algorithms and sophisticated analytics developed with key stakeholders (e.g., educators and administrators), and vetted by psychometricians and applied scientists. A rigorous psychometric approach was used to develop the LearnPlatform grading rubric, which educators use to subjectively evaluate edtech products and differentiate effective and ineffective technologies. Further, rigorous scientific approaches were used to develop the analytics engine that drives the IMPACT™ tool, which leverages multiple research methods and flexibly adapts the specific research design (i.e., control, comparative, or correlative) based on the data inputted into the system.
How does IMPACT™ work? What is the methodology?
Once data are uploaded, the advanced IMPACT™ analytics engine generates insights into product implementation and impact. A backend algorithm is used to group students into natural usage groups and identify patterns across student groups with differing levels of usage. A quintile analysis then partitions students into subgroups based on levels of prior performance and examines the effectiveness of the product to improve education outcomes for each performance group (note. product effectiveness is only examined within Outcome Analyses). For example, a Usage Analysis partitions students based on the extent to which they used the product, and then examines product effectiveness for each group (within Outcome Analyses). In Outcomes Analysis with a Comparison Group, IMPACT™ analyses account for differences such as student demographics (e.g., gender, ethnicity, socioeconomic status), grade level, and prior achievement in determining the effectiveness of an edtech product (i.e., covariates). A Cost Analysis provides information on the total cost of ownership, cost-effectiveness, and amount of money spent on different usage clusters and fidelity groups. The on-demand analytics dashboards display edtech product insights in a transparent and easy-to-use way.
