Student engagement on education-technology apps should not be equated with learning.
That’s one of the core messages of a new report released by the American Psychological Association. The report, written by a multidisciplinary panel of psychologists, synthesizes evidence across a wide range of digital learning apps and offers 10 evidence-based recommendations meant to help families, educators, policymakers, and product developers ensure that technology promotes meaningful learning.
“The real test of learning happens after the app is closed,” says the report, released Sept. 3. “Students may perform well within a tool but fail to retain or apply what they learned elsewhere.”
The report recommends that the education and tech communities take the following steps:
- Identify learning goals and outcomes when using ed tech, and judge apps by those metrics instead of by the level of student engagement or enjoyment;
- Evaluate what other forms of learning technology is replacing;
- Choose tools built on research-based learning strategies; and
- Make adult involvement a significant part of implementation—because ed tech is most effective when learning is supported by human relationships.
The report comes as school districts grapple with a growing backlash against students’ use of technology and its correlation with behavioral, mental, and academic challenges. This month, New York City public schools joined the list of districts restricting school screen time for younger students.
In an interview, Nicole Barnes, APA’s executive lead psychologist for education, discussed the research on ed tech, what the organization is recommending, and her view of the role of screen-time limits in schools, a policy option favored by some parents and advocates.
This interview has been edited for length and clarity.
What’s APA’s take on the screen-time conversations happening in schools right now?
What the science tells us is that the choice is really not between screens or no screens. The choice that we’re making is whether a particular learning experience is better with that technology and what students give up when that technology is used.
When we look at screen time as though it is its own independent bubble, that alone cannot tell us whether a child is using that time to solve a problem or reading deeply. It also can’t tell us whether that child is just passively watching something or switching around among distractions.
What does that tell you about steps schools should consider taking?
Schools need to start asking the harder questions. It’s really easy to just say we’re going to put a ban on something, but to me, that is almost usurping the more important questions: What is this technology doing? When is it being used?
Is it replacing something that we know is really important to learning and development, like discussion, peer interaction, play, sustained reading, hands-on learning, or sleep?
What is the biggest takeaway you want educators to draw from this report?
For a long time, ed-tech companies were able to present engagement metrics in a way as though they represent learning. In the science, we know that there is a big difference between engagement and learning, and we need to stop conflating the two.
An ed tech product can hold a student’s attention. It can generate really impressive usage numbers, and it can still fail to produce the knowledge and skills that will last.
What is the message schools should be delivering to developers of these products?
I’m hoping that as a community we are going to start demanding [answers] of these educational technology companies. Not just: Did this student like it? Did they succeed while using it in that app?
But more importantly: Can they remember? Can they explain? And can they apply what they’ve learned after this tool is taken away or after we put the screen down?
Many district and school leaders may agree, but also wonder how they should evaluate these products. What’s your advice to them?
They need to look for the “show-me” moment. Show me that what you can do in the app, you can do outside of the app—whether that’s explaining the idea, solving a new problem, or applying the skill in another setting.
Success inside this highly supported platform may only be showing that the student has learned to navigate the platform, and the real test is whether they can transfer what they’ve learned inside that platform to something else. That is when I will be convinced that there is evidence of learning.
The report differentiates between ed-tech apps and general-purpose ones. Why is that distinction important?
We can’t treat all technology that is used in an educational setting as an educational technology tool. The purpose of an ed tech tool is to foster learning in some way. There are other technologies that are being used in schools that are not for that purpose, and more importantly, there are other technologies being used by youth, regardless of if they’re in schools or at home, that are not for that purpose.
The ones we’re talking about here are the ones that promote themselves as having some type of learning gain, and what we are asking those companies to do, through external validation, is prove that those learning outcomes are in fact what they say they are, because we’re not protecting our youth if we don’t.
Do you think school communities and parents are making these distinctions?
There are a lot of ed tech tools out there that are great and actually have done the hard work, and it’s not fair that they are put in the same bucket as those that are not. Ed tech has helped learners in ways that are exponential, and especially for students with disabilities or English-language learners.
They’ve transformed learning for those populations in very good ways. The lack of regulation has allowed a marketplace where those companies that are not doing it well are allowed to compete at the same level, and that’s just really not in the service of protecting our youth, which is what we should be about.
Where does AI fit into the conversation?
AI is being built into these tools. The banning is not capturing the nuance that’s really required. The goal here is to keep the student doing the thinking.
AI needs to be used to strengthen learning. We know that when AI helps you retrieve information from memory, when it helps you revisit material over time, when it gets you to explain your reasoning or compare related problems, or when it tailors feedback in the way that is the most helpful, or apply knowledge in new contexts, then AI is a powerful learning tool.
It’s less about, do we use it or don’t we use it? It’s more about the how.
It seems like learning science has become a buzzword in ed tech. What do you think about that?
There is a difference between saying that your product is rooted in something or has foundations in something, and saying your product will lead to these outcomes. That conflation is what’s going on right now and what needs to be separated. It’s a good step that your design features are grounded in what we know helps students to learn and then sustain those gains over time.
But that doesn’t mean that you don’t also need to be able to, with independent external validation, prove that those claims that you are putting forth to the public are in fact true. I see it as a “yes, and.” They need to be rooted in the science, and they need to prove that they actually produce the outcomes that they say they do.
Where should educators start in trying to hold tech to these kinds of standards?
Having a heightened sense of awareness and intentionality with what students are being asked to do, and how they are spending their time when they’re on these apps would be a really great first start that schools can easily do and would not require them to go much outside what they already do in their fiduciary responsibility to educate students.
A necessary second step—I don’t think this will be on any one particular school; this will be probably more at the state and district level—is to contain the Wild, Wild West that’s going on right now. There does need to be a sheriff in town coming up with what those standards are going to look like across the industry, what the procurement rubrics or criteria are going to look like.
