Requesting Usage Data for Running Rapid-cycle Evaluations

Getting Data From Providers (General Talking Points)

Understanding What's Working and Driving Improvement

Our district works with LearnPlatform to manage and evaluate our edtech. They support us in making edtech evaluation part of our standard practices – to ensure we are implementing products well and getting the most value from our investments for our students and teachers.

In order to do this, we need to understand how and to what extent our students and teachers are using your solution. 

How We Will Use This Data

The usage data will be used to run rapid-cycle evaluations (RCEs) within LearnPlatform to better understand how products are working, for which students and teachers, and under what conditions. RCEs help us generate evidence needed to:

• Build on our most successful implementations to advance best practices across our district.
• Identify needs for additional teacher support and/or professional learning refinement with certain tools.
• Understand where usage is not meeting expectations, so we can begin to explore why and uncover any barriers to use. 
• Discover the impact of program changes and interventions over time on specific groups.
• Generate evidence needed to comply with local, state and federal regulatory and reporting requirements.

The data will be put through LearnPlatform’s data-secure, rapid-cycle evaluation engine, IMPACT™,  alongside other data from our district (like demographic information and assessment scores). Evaluating program usage data alongside other data helps us determine how to improve implementation for better results.

Have questions? Please email support@learnplatform.com.

Getting Data from Providers (Suggested Email Template)

Hi [name],

As you know, our district wants to better understand how our students and teachers use your product, and how we can better support that use. In order to do that, we need to access our usage data – here are some details about how the data will be used. Below, you will find more details, including data formatting standards and privacy/compliance information.

Data Format: Data coming to LearnPlatform for use in rapid-cycle evaluations should meet the following basic criteria:

• User-level information (vs class, school or averaged data).
• All external identifiers, including district or third-party IDs included.
• One row per user..
• At least one categorical usage metric (time, lessons completed, etc.).
  • Multiple usage metrics are encouraged. 
• Include data for only the requested time period.
• Data is shared in a .csv formatted file.

Sharing Data: We use ExaVault for receiving files that include personally identifiable data (PII). This URL [link] can be used by your team to share files directly to LearnPlatform. There is no password required.

If you prefer to use your own internal systems to share data with LearnPlatform or if you would like to automate data sharing, please let us know, and we can determine the next best steps.

Data Use: Based on the parameters of this engagement with [organization name], we anticipate that the data you share with us will be used in the following ways: usage reports, outcomes reports, and on these general timelines [quarterly]. 

We are planning to support [district name] in data collection starting on [date]. Who can we work with on your team to discuss the requirements, answer questions and collect the data?

Thank you, and we look forward to partnering with you on this work!

Getting Usage Data from Your Own Team (Recommendations)

It’s possible you may need support in getting usage data from team members inside of your own organization. Here are some tips to help you out!

Key talking point: We are working with LearnPlatform to help us identify areas where our implementation of edtech programs is going well, and where there are opportunities to have better, more detailed conversations about what’s working for your staff and students. We will only be asking you for the specific data that we need to conduct this work. 

Recommendations and Tips

• Do:
  • Share often and share openly. By this, we mean that it’s easier to proactively discuss the value of RCE work with your team and other stakeholders.
    • Consider using LearnPlatform resources before the implementation of RCE work begins, and consider which groups of people may have specific questions or concerns: 
      • District leadership (cabinet level): Provide specific information about your plans, and how the results may be used. Encourage conversations and discussions earlier vs later, and make sure to share initial reports and results for feedback and to increase engagement.
      • Your academic teams: Provide them with information about how RCE work can help them to improve their teaching practices. Share that this work is not punitive, but rather a formative way to get data-driven information so that they can all have better conversations about what’s working, and for whom. Invite them to be an active participant in this process as early as possible.
      • Your data and assessment teams: Reinforce that RCE work with LearnPlatform is there to increase capacity and effectiveness of the work they already do; and that no one’s position or responsibilities will be threatened in any way. Be specific about what data files are going to be used for; and set the tone early on. Make sure your team understands that LearnPlatform doesn’t make any decisions about how to interpret or act on RCE results. 
• Don’t wait too long and don’t leave it to chance! The sooner you can speak with your teams and promote understanding of this work, the easier it will be to work with them now and in the future. 

Usage Data Requirements for Solution Providers

LearnPlatform allows education organizations to load product utilization data into their accounts to run rapid cycle evaluations (RCEs) on edtech solutions in use by teachers and students. These evaluations help districts to maximize impact of program use, adjust implementations as needed and ensure that the correct students and teachers have access. You may receive requests for usage data from districts to run these RCEs through LearnPlatform’s IMPACT engine. Click to learn more about LearnPlatform’s RCE with IMPACT <LINK>. 

When districts ask for usage data for use with Learn Platform’s IMPACT engine, they are looking for user level data on how and how much the program was used in whatever format your product reports those values. Here are the content and format requirements your district partners will need data files to conform to in order to complete this research:  

File Type: CSV

File Naming: The file name should include the name of the product and the evaluation time frame. E.g. SmartMathUsage_01-01-2020to12-30-2020

File Format: The data should be delivered in a CSV where each row is a unique record or case (almost always a student), and each column is a variable on which data are collected for each record. 

Data Requested: User-level (most often student-level) program data including usage and/or performance metrics for a given timeframe. 

Record Format:

 *Primary Key = Product Identifier, NCES ID, Student/User Identifier, Period Type, Period Value, Metric Type 

File Examples*: 

Example A: In this example, usage metrics reported are minutes on the system and count of questions. The unique student identifier in use is district ID numbers. 

Example B: In this example, usage metrics reported are logins, count of modules completed and total session time. The unique student identifier in use is student name. 

The examples above are meant to illustrate the different ways data can look, from various providers. LearnPlatform understands that your solution is unique and we have committed to building research tools that are flexible. If you have questions about your program data and how it fits into this template, please contact us at support@learnplatform.com. 

Automating Data Transfers 

Automating data transfers will save time for both you and researching districts. If you’re interested in automating data transfers, please contact us at support@learnplatform.com.

(For Providers) How Do I Claim My Company Page?

If you are provider, feel free to check out our article here on joining LearnPlatform and claiming your company's page.