Professor Umar Toseeb Hello, welcome to the Papers Podcast Series for the Association for Child and Adolescent Mental Health, or ACAMH for short. I’m Umar Toseeb, Professor of Psychology. My research focuses on special educational needs and mental health in childhood and adolescence. All listeners to this, and indeed any of ACAMH’s podcasts, are eligible for a free CPD certificate. Do please visit acamhlearn.org for details of this, together with information on how you can access hundreds of hours of free talks, lectures, interviews, all of which you can also get free CPD certificates for. The web address is acamhlearn.org, that’s a-c-a-m-h-l-e-a-r-n.org. Today, I’ll be speaking to Professor Charles Hulme, Emeritus Professor at the University of Oxford, and Professor at Oxford Brookes University, one of the co-authors of the paper, “Oral Language Intervention in Late Primary School Years is Effective Evidence From a Randomised Controlled Trial,” published in the JCPP. Charles, thank you so much for joining me. Professor Charles Hulme It’s a pleasure, very nice to talk to you. Professor Umar Toseeb And shall we start with some introductions? Do you just want to give us an introduction about yourself, but also some of the co-authors on the paper? Professor Charles Hulme Yeah, sure. My name’s Charles, my main interest now, in the last stages of my career, are in interventions, particularly interventions to improve children’s language and reading skills, but I’ve been more broadly studying the nature and causes and treatments of children’s learning difficulties for the last 40 odd years, I suppose. My two co-authors, Rosanne Esposito, is Associate Professor in the Institute of Education at UCL in London and is my PhD student at the University of Oxford. The other author is Arne Lervag, who is Professor of Education at the University of Oslo. Professor Umar Toseeb Thank you, and before we go into the intervention and the paper itself, let’s just get a bit of a background. What do we know about the importance of language for children’s development? Professor Charles Hulme I think we know a lot. I’m on a, sort of, crusade to persuade people that language is really the foundation for the whole of education and for many other aspects of children’s development. We know that children with language difficulties are at higher risk of social problems, they’re at high risk of reading and other forms of educational problems. Longer term, they will have reduced life chances and lower expectations of employment in adulthood. I think you could make a case that it’s plausible that language is a causal risk factor for the development of many other downstream adverse effects, including social difficulties, mental health difficulties, poor educational outcomes and poor adult outcomes in terms of wellbeing and employment prospects. Professor Umar Toseeb And is there an intersection with socioeconomic status? Professor Charles Hulme There’s a very, very strong relationship between language difficulties and socioeconomic status. In some unpublished data we’ve got from our LanguageScreen app, we’ve shown very strong relationship between levels of deprivation within a school, measured by the proportion of children in receipt of free school meals, and standardised scores on our language assessment. The difference between the most advantaged schools and the least advantaged schools is somewhere around 11 standard school points, so that’s over two thirds of a standard deviation. So, the differences between children in schools with high levels of deprivation and children in schools with low levels of deprivation are startlingly large, actually. Professor Umar Toseeb And when you say "language," and the conversation that we’re going to have, shall we talk a bit about what you mean by language? So, structural language or receptive, expressive, what are we going to be talking about? And the evidence that you’ve talked about so far, which aspects of language is it about, or is it about broadly language? Professor Charles Hulme That’s a really good question, Umar. So, language is our ability to understand what’s said to us and to express our thoughts using language, and those are often referred to as “receptive” and “expressive language skills.” I actually believe that in the earlier school years, at least to a first approximation, language is what we might call a "unitary trait," you either have more or less of it, and that tends to be reflected in measures of both receptive and expressive language skills. Professor Umar Toseeb Thank you, and what do we know about language interventions? So, do we know that when language interventions work for children’s development and language as a whole during childhood and adolescence? Professor Charles Hulme There’s a growing literature showing that language interventions can be effective. Language is quintessentially a skill that we learn, and we know that structured interventions, such as the intervention we’re going to talk about today, which we’ve called "OLLI," and our other language intervention called the "NELI programme," or the “Nuffield Early Language Intervention programme,” there’s now very good evidence, I think, from numerous robust RCTs that language intervention can be effective in producing meaningful improvements in children’s language understanding and production. Professor Umar Toseeb I was speaking to someone a few weeks ago about what a language intervention should aim to achieve, and we were talking specifically about children with developmental language disorder, and the conversation was on the topic of, "Should the aims of a language intervention be to improve that child’s – a given child’s language to what’s age appropriate, or the standard for that age, or should it be to give that child enough functional knowledge of language to be able to access some aspects of schooling, but not necessarily get to the same stage as age appropriate peers?" What are your thoughts? Professor Charles Hulme I don’t think there’s any really clear absolute cut-off between children who would be diagnosed as having DLD and children with less severe language difficulties. Where you place the cut-off between DLD and other lesser forms of language weakness is, to some degree, arbitrary. My view is that you can move many, many children who we identify as having significant language difficulties into the normal range, by which I mean within one standard deviation of the average level of language skills for their age. I think that, sort of, help, however, probably needs to be thought of as something that would ideally be provided on an ongoing basis over a fairly long developmental period. Ideally, I would say, starting in preschool and going through a number of years in primary school. But with such fairly consistent help, which can be actually delivered at quite low cost, we can make very meaningful improvements in children’s language skills. Professor Umar Toseeb Okay, and so you’ve talked about how language interventions can help to improve language skills, and that’s useful in itself, but I suppose the other rationale for trying to improve children’s language would be to improve their development in other aspects, so educational, cognitive and social development. Do we know that language interventions improve outcomes or development in other areas, other than language itself? Professor Charles Hulme Yes, they do. These effects seem to transfer to other aspects of development, but I would say that this is an area where we badly need further studies. In one of our previous very large-scale randomised trial in over 200 schools, we showed that the NELI programme was actually associated with improvements in children’s behaviour in school, which was quite a strong and striking finding. A number of studies have also shown now that language interventions can transfer to improve decoding in reading and also to improve reading comprehension. Professor Umar Toseeb Excellent, and what is the evidence gap that you’re trying to address in the paper that we’re going to talk about? So, you’ve – to me, it seems like the evidence for language interventions, helping to improve language, is there, based on what we’ve talked about. So, what is the evidence gap that you’re trying to address? What do we not know? Professor Charles Hulme I guess the particular focus of the paper we’re going to talk about is delivering language intervention to older children, broadly, children between the ages of nine – eight and nine-years-told. The vast majority of the evidence concerning language intervention comes from studies of much younger children, typically kids between, say, four and six years. And I guess we wanted to demonstrate that interventions delivered in the later primary school years can still be effective, and in this study, we showed quite clearly that they could be. Professor Umar Toseeb Yeah, I was going to say, I think that if I was to hypothesise before I read the paper whether a language intervention would work later in primary school, I would say, if it does, it would be – the effect sizes would be much smaller than they were – would be in earlier primary school and early life, and we can talk about those. And I think part of my rationale for guessing that, or predicting that, would be, if you’ve got children who have struggled with language for the first parts of primary school, delivering an intervention in the later parts of primary school would require then to catch up and continue to develop language, because I imagine it’s – well, you know, it’s a continuous process. Is that what you would have expected? Would you – did you expect it to be effective but not as effective, or what were you thinking going into it? Professor Charles Hulme So, I’ve got a different view of this to you, Umar, I think. Actually, in some of our work, we’ve found it’s quite difficult to get meaningful improvements in language skills in very young children, I’m talking about preschool children. Sometimes we get them, sometimes we don’t, and our view is that many of those children who are, say, three-year-olds with language difficulties, also may be highly immature, might have some other cognitive limitations, and delivering the language intervention to some of those children can be quite challenging. The study we’re going to talk about grew directly from a previous study we’d done, published in 2010, I think. And in that study we got effect sizes in the .3 to .4 range. That’s what we got here, and that’s, basically, what I was expecting. Professor Umar Toseeb Okay, excellent. Thank you, and let’s talk about the intervention, so it’s the OLLI intervention. Can you just talk us through the intervention as it was intended? So, like, things like what are the key ingredients of the intervention, the dosage, and those kinds of things? Professor Charles Hulme So, OLLI is based very directly on an earlier study of language intervention that we published in 2010. That was a paper – a piece of research conducted at the University of York, led by Paula Clarke. Like all of our interventions, it’s a 20-week intervention, it’s fully manualised, and it’s designed to be delivered by trained Teaching Assistants, but we also train the Teachers so that they understand the nature and content of the intervention. It’s designed for children who are, if you like, at the bottom of the distribution of language skills. So, we screen all the children in each classroom, and in this study, we selected the six children in each classroom with the weakest oral language skills. We do that screening with a tool we’ve developed called “LanguageScreen,” which is an automated app that can be used by any adult really, to assess a child’s language skills in roughly ten minutes. It’s automated, it gives a highly reliable and valid assessment of a child’s language skills. And then the OLLI programme consists of small group and individual sessions over 20 weeks. There’s a large component of it involves active listening, listening to stories, talking about stories, direct training of vocabulary, and direct training of what we call “figurative” or “non-literal language skills.” We teach the kids jokes and we also teach them the similes and metaphors. So, you know, teaching a child, “What does it mean if we say, “It’s raining cats and dogs”?” You can have a long and quite interesting discussion with children about why on earth that’s an expression we use in British English. There’s a big emphasis, also, on getting children to produce their own narratives and giving them structured, what we call, “scaffolded supports” in speaking. Professor Umar Toseeb And I suppose it seems quite reasonable and straightforward, but if you could just tell us what your theory of change was? So, you – is it language intervention, and then, what does that lead to, which then leads to the outcome? I think is what I’m saying. Professor Charles Hulme I think the theory of change here’s very simple. I mean, I always say to people, “If you want to teach people to play tennis, get them on a tennis court and teach them tennis. If you want to teach children language, well, directly teach them language.” Directly teach the meanings of words, you directly give children stories to listen to, you discuss the stories with them. Importantly, I think there’s quite good evidence, some of this comes from Maryellen MacDonald in the US, that language production is particularly important as a means of learning language. So, if I’m trying to learn French, it’s much better for me to try to speak French, get it wrong and be corrected, than simply to listen to a lot of French that people are talking – when people are talking. So, I think the theory of change is a very simple one, that we directly teach a set of skills, the underlying components of language, if you like, to the children, and we expect those skills to improve. Professor Umar Toseeb Thank you, and let’s go onto the method itself. So, you’ve talked about what the intervention is and what it was intended to be. Now tell us about the sample that you used and how the intervention was delivered to and – who it was delivered to and what the outcomes were that you were interested in. Professor Charles Hulme I should say this study was conducted by Rosanne Esposito as her PhD with me at the University of Oxford. I think what Rosanne did here is pretty remarkable for a PhD. She recruited 33 schools that contained 50 classrooms, and we then got staff in those schools to screen all children in the participating classrooms using our LanguageScreen app, so that screening involved over 1,400 children. And from those children, we selected the bottom six children in each classroom as potentially suitable for intervention, and then on a within classroom basis, we randomly allocated within classrooms children to either receive the intervention or to not receive it and to be in the control group. We trained – when I say “we,” Rosanne trained the Teaching Assistants and the Classroom Teachers in the nature of the OLLI programme, and the Teaching Assistants then delivered the intervention over a, roughly a 20 week period. We didn’t get perfect delivery. There were a whole set of challenges to this associated with COVID. I should say that actually we started this study first and then abandoned it because of COVID, and then had to re-recruit schools and start all over again a second time. But even the second time, there were still some hangover effects of the COVID pandemic, which made it quite difficult. But that’s the design, recruit a lot of schools, screen all the children and then train school staff to deliver this programme over a 20-week period. Professor Umar Toseeb I have a question about your randomisation. Why randomise in the way that you did? So, every time I’m trying to propose an intervention or an evaluation of an intervention, I always avoid randomisation at that level, just because of potential contamination effects. I always say, “Let’s randomise at the school level,” because then you can avoid, to some extent, or to a lot of extent, the contamination. Whereas, if you’ve got kids in the same classroom, three who are receiving the intervention and three who are not, which is I think what you’ve described… Professor Charles Hulme Yeah. Professor Umar Toseeb …real risk of contamination. Professor Charles Hulme So, that’s a great stats question. The first reply is that if you randomise at a class or school level, you have a major reduction in power, and as you know, that reduction is proportional to the degree of the within cluster similarity between children. What you say about contamination is a worry, so I would say the effect sizes you get from a within class randomisation design like this are, if anything, an underestimate of the true effects. But actually, I don’t really believe in this stuff about contamination, or at least, I believe it’s possible, but if all we needed to do was to say to people, “Oh, there’s something over here and it might help,” we don’t actually believe people have the time to do the intervention. You know, the intervention takes a lot of work and the fact that kids are in the same classroom as kids who are getting the intervention, you know, it’s very, very unlikely that they will receive anything approaching the intervention. And of course, we did emphasise to the school – participating schools, “You should only be giving the intervention to the designated children, because this is a research study.” So, I would say we’ve got very good estimates of the effect sizes here that might be a tad conservative. Professor Umar Toseeb And in terms of the outcomes, what was your primary outcome, and then you had a number of secondary outcomes, what were they? Professor Charles Hulme Yes, so our primary outcome was improvements on language ability measured by our LanguageScreen app. The other important outcome of the study was a measure of children’s written expression, their ability to write a passage based on a prompt that they were given. Professor Umar Toseeb And what did you find? Professor Charles Hulme We found substantial improvements in children’s language ability with a d of about .38, and, also, we were very pleased that we found equivalent, or slightly larger, improvements in a measure of children’s written expression, with a d of .42. Professor Umar Toseeb And how does that translate in terms of, I don’t know, maybe months of progress… Professor Charles Hulme Hmmm. Professor Umar Toseeb …I think is what’s used in education evaluation circles, but I suppose in your paper, you talk about “educationally meaningful,” so do you want to tell us a bit about that? Professor Charles Hulme Another crusade I’m on is to try to move people away from using “months of progress,” because this is a deeply flawed measure, but people use it. If you wanted to translate these effects into months of progress, they’d probably be in the range of four to six months additional progress, I would say. So, no, not six, probably three to five additional months progress, I would say. In terms of interpreting effect sizes, I’m very influenced by a paper by Matthew Kraft, reports a meta-analysis of over 1,000 educational interventions. And in that paper, Kraft shows that most of these interventions have, by conventional standards, tiny effect sizes, of less than .1, and many, many have effect sizes of less than .2. Probably most have effect sizes of less than .2. In our paper, we have effect sizes around .4. They’re really very strong. Effect sizes of that sort, you know, moving children’s skills by .4 of a standard deviation, if those effects persist, even not completely persist, but if we can produce effects of that sort of size that have any degree of persistence, they’re going to make really educationally meaningful improvements in children’s skills, I would say. Professor Umar Toseeb And did you look at different groups, or was there not enough power to do that? So, did you look at certain disadvantaged groups that you might expect to benefit more from such an intervention? Professor Charles Hulme We didn’t. Our focus here was very much on the, what we call the “average treatment effect.” In some of our earlier work, we found slightly stronger effects for children from English as an additional language background. that tends to be more in younger kids. These kids have been in school – in our case, they’re in year three, year four, they’ve been in school quite a long time, so differences between kids from EAL backgrounds and non-EAL backgrounds tend to reduce at that age level, anyway. By definition, the kids getting our programme tend to be children from less advantaged backgrounds. So, I would say these programmes are particularly valuable for children from relatively disadvantaged homes, who are most likely to be identified by us as having language weaknesses. And we get no evidence in this study of any – again, without getting too statistical, there were no differential effects of the intervention as a function of initial language level. In other words, it helps everybody to about the same degree, wherever on the distribution of language ability you are to begin with. Professor Umar Toseeb And with lots of intervention studies, and I see this a lot with, I won’t mention the names, but, like, a large education evaluation funder, where a lot of the time, it’s pre-test, intervention and then post-intervention, you do the outcome measure, which is what you’ve done here. And my question is always, “What does this mean for the longer term?” So, if you tested these kids 12 months, for example, after the intervention had finished, would you still expect to see the beneficial impacts of the intervention, or what would you expect to happen? Professor Charles Hulme That’s a really, really good question, and I think, again, a big gap in our knowledge in this area is a shortage of really long-term follow-up effects. But let me just share some results from a long-term follow-up of our NELI intervention. We followed up a sample of children from our 2021 paper in JCPP who had received the NELI intervention two years after the intervention had been delivered. And what we found there is quite remarkable, which is persisting improvements in children’s oral language skills, and, also, improvements in their single word reading and in their reading comprehension ability, two years after the intervention had finished. So, the intervention finished when kids were about six, and when they were eight-years-old, they were still showing meaningful improvements in language and single word reading and reading comprehension ability. So, I’m moderately optimistic that the OLLI programme will produce assisting improvements. In the predecessor of this study, the Clarke et al. study that we published in Psychological Science in 2010, the effects there persisted six months after the intervention had finished. I think part of what we’re doing in these interventions is actually – this goes beyond the data, so this should maybe come with a health warning, but I do believe that what we’re doing in these interventions is changing these children’s metacognitive skills. We’re taking kids who have language weaknesses and teaching them that if they engage with language learning, they can learn language and succeed, and they get a lot of reinforcement from getting better. And in the OLLI programme, one of the features that Rosanne built into it was teaching children to ask when they didn’t understand something. So, if you’re a child with language learning weaknesses, and you’re, you know, maybe not that happy about the fact that you find language hard, you sit quietly at the back of the classroom and keep your head down and don’t say anything, that’s not going to help you. If you regularly say to an adult, “I didn’t understand what you said,” or, “I don’t know what that word means,” that gives you the – a mechanism by which to engage with language learning in an active way and make long-term progress. I’m optimistic that these effects might be moderately durable, but that’s something I’d love to do more research on. Professor Umar Toseeb And we know that in England, well, I imagine in lots of places, not enough resource that goes into schools, and parents of children who are struggling at school might want to help their child. Is it possible to adapt this specific intervention for it to be delivered by parents themselves rather than in the school setting? Professor Charles Hulme I think you can do that, but I think there are big challenges. Because, as we’ve said, language difficulties are most prevalent in children from relative depri – relatively deprived homes, and their parents may be not rich in terms of time, other resources, and may themselves not have the best language skills. But we have published some research showing that you can train parents to deliver language intervention. That study used a paired book reading approach. I think parents can be engaged to help with this work, and one of the things we’re currently working on is trying to create very positive links between the language interventions that are delivered in school and providing parents with information about what’s happening in the classroom, and providing them with exercises, games that they can play with kids, to reinforce the language work that’s happening in school. So, I think parents can be involved, but I really would make a plea for thinking seriously about making oral language work a part of the national curriculum. All schools ought to be doing work on oral language. Oral language is the foundation for education, and it happens best, I think, in school, rather than leaving it to the parents to deal with. Professor Umar Toseeb And that brings me my next question, so you might just want to expand on that if you want. It's, what are the implications of this piece of work and your work more broadly for people who work in education? Professor Charles Hulme I think the implications here are massive. For many, many years, I don’t think people in education have recognised the critical role that oral language skills have for pretty well everything that happens in formal education. Education mostly involves the people being taught listening to what’s being said to them, understanding it and then re-expressing that knowledge in assessments, which are usually measures of verbal comprehension of material that you’ve learned. So, language is fundamental in all of these respects. Language is, more broadly, also a fundamental social development. Children with language difficulties often have a whole range of social problems, difficulties with friendships, difficulty with relationships with their Teachers. So, I would see a very strong case. Nobody doubts that when children go to school they should be taught to read. In fact, you could say that in Britain, we have an obsession with schools teaching kids to read. Now, I think that’s quite a good obsession to have. Everybody should be taught to read, and we should ensure that everybody had adequate reading skills to support the rest of their education. What I would say on top of that is we should have an equivalent obsession with teaching all children oral language in the context of school. Because actually, oral language is the foundation for the whole of literacy development, reading in the sense of decoding, reading in the sense of comprehension, and literacy in the sense of being able to write. So, I would say language needs to be incorporated into the school curriculum, and our study shows one way of doing that. Our other studies with the NELI – the various NELI programmes also show that we can get very good effects from school-based language enrichment and language intervention. These are low cost, they’re enjoyable for the kids, they’re enjoyable for school staff. There’s absolutely no reason for this not becoming part of our national curriculum, I would say. Professor Umar Toseeb And what should we look out for in this space, coming up from you and your group? Professor Charles Hulme I think we’re working very hard, really, to get these sorts of interventions adopted at scale. We’ve been very fortunate that we’ve had funding for the last five years from England’s Department for Education to rollout the NELI programme in roughly two thirds of all English schools. That rollout has been effective. We’ve got very clear evidence that schools that have received the NELI programme have produced improvements in their – those children’s language skills. What we really want to do is to persuade the government to provide funding for the Teaching Assistant time that’s needed to deliver these programmes, and really get them embedded in the school curriculum. Professor Umar Toseeb Finally, what’s your take-home message for our listeners? Professor Charles Hulme Language is everything, language is the heart of education, and language is a very significant aspect of child development that needs to be nurtured and facilitated. Professor Umar Toseeb Thank you, Charles, and thank you for the very insightful conversation and discussion. I’ve learnt a lot. Thank you. Professor Charles Hulme A great pleasure. Nice to talk to you, Umar. Professor Umar Toseeb For more details, please visit the ACAMH website, www.acamh.org, and Twitter @ACAMH. ACAMH is spelt A-C-A-M-H, and don’t forget to follow us on your preferred streaming platform, let us know if you enjoy the podcast, with a rating or review, and do share with your friends and colleagues.