Professor Samuele Cortese Hi, my name is  Samuele Cortese. I am an NIHR Research Professor,   Professor of Child Adolescent Psychiatry at  the University of Southampton in the UK and,   also, a Professor of Child Neuropsychiatry at the  University of Bari in Italy and at the University   of New York in the United States. So, the  presentation focuses on “Interpreting the   Evidence Base, with Some Examples from ADHD.”  So, before moving to the presentation itself,   let me disclose my possible conflict of  interest, in relation to this presentation. Right, so, let’s move now to the focus of the  presentation. So, the purpose is to discuss the   different methods that are available to synthesise  the evidence that informs our clinical practice,   using examples from ADHD. And I think it’s very  important to be aware of what really evidence is,   because I think, usually, every Clinician in their  clinical practise, every day, they will mention   the term ‘evidence’, saying, you know, “We need  evidence, there is evidence, but what is really   evidence, and how can we synthesise, how can we  summarise the evidence?” So, I will provide some   example of some high-level evidence synthesis  approaches, using studies in the field of ADHD. So, I guess at least some of you may be  familiar with this figure which represents   the different levels of evidence. So, first  of all, when it comes to the term ‘evidence’,   we need to be aware that there are different  levels of evidence. At the bottom we have   evidence which comes from case reports or  case series, which, arguably, is biased,   because it’s based just on an observation  and the study design is not rigorous,   of course. And, going up, we have case-control,  cross-sectional, case control or cohort studies,   but really when it comes to evidence  supporting the use of treatment,   really the best design, they call ‘standard  design’, is the randomised controlled trial. Of course, we may have findings wh – on  the same treatment, in the same population,   which are different across different  randomised controlled trials and hence,   this is where the needs to synthesise the  evidence comes, because we cannot rely just   on one individual trials, when there are many  trials out there on that particular treatment,   for that particular population,  for that particular outcome. So,   in that case, of course, the highest level is  represented by the synthesis of this evidence,   which is conducted typically in  systematic reviews with meta-analysis. There are different types of meta-analysis and I  will mention some of them in this presentation.   So, just to clarify, systematic review is  really a collection of studies on the same   topic. Meta-analysis is simply a statistical  approach, which allow us to pull together   the data which comes from these different  studies retrieved via systematic review. Actually, this is quite outdated, this  figure, because now we note that at   the top of these levels of evidence is  what we call the ‘umbrella review’. So,   umbrella review is a systematic review, which  can be done in a qualitative or quantitative way,   of meta-analysis and systematic review, so is  a level higher. We are now in a stage whereby   for the same types of intervention  we have different meta-analysis,   so we need to pull together data  from these different meta-analysis. Right, so, I will give you now an example of  different types of evidence synthesis. So,   let’s start with the standard meta-analysis, which  is also called a ‘pairwise meta-analysis’. So,   this is a meta-analysis of studies which  compare a specific intervention to a control,   typically, and so this is why they are  called ‘pairwise meta-analysis’. So,   the question is, does this treatment  works bet – work better than the control? Of course, if we have ten studies that  have been conducted on this topic,   ten trials, and, let’s say, four tell us  that there is no difference and six tell   us that there is a difference, that  the treatment is superior to control,   what shall we conclude from this body of evidence?  It will be unwise just to conclude that, you know,   we rely on the six studies because they’re the  majority, because the six studies may be smaller,   they may have methodological problems, so  they may not tell us really how things are. So, what shall we do? Thanks to the meta-analysis,  we pull together the data from all these studies   and we give more weight, more important,  to the studies which are the best. So,   let me show you now an example of this  pairwise meta-analysis, highlighting a   specificity which we need to consider in child  adolescent psychiatry, in particular, also,   in the case of ADHD. Let’s consider the  role of non-pharmacological interventions   for ADHD. We may wonder, for instance,  a specific type or non-pharmacological   intervention is behavioural treatment or parent  training. We may wonder, is parent training   better compared to a control condition, in  terms of improving the symptoms of ADHD? So, what we can do is, let’s find out  all the trials that have assessed – that   have compared parent training with control  condition and have measured the symptoms of   ADHD. Let’s pull together the data from  all these trials to have a conclusion,   to conclude as to whether parent  training works or not for ADHD. However,   a specificity in the field of ADHD and more  general developmental psychopathology is that   we may rely on different types of raters of these  symptoms. To measure if there is an improvement,   of course, we measure the symptoms at  the baseline and at the end of the trial,   but these symptoms can be provided by parents,  Teachers, self-reported, and so on and so forth. So, in this pairwise meta-analysis, we found that  if we look at the symptoms provided by parents,   which we called ‘most proximal’, because they are  proximal to the delivery of the intervention, we   have a difference, a significant difference. And  this is the way we represent the meta-analysis;   this is called a ‘forest plot’, because  it looks like a kind of forest, and   vertical. So, the way it works is the  following. Each line represent a trial,   which has compared, in this case, parent training,  behavioural intervention, versus control. Each – for each trial you see a dot here,  which is the effect size. The effect size is   the magnitude of the effect and the effect  size can be on the life – left hand side,   on the right hand side. Everything which is on  the left means control is better than active   treatment; everything which is on the right  side means active treatment is better than   control. However, crucially, you don’t have  just to look at the effect size, but, also,   the confidence interval. This is, as you  may remember from your statistic class,   95% of certainty, the real effect is along  this line, so it can be here, here, here. So, what do we conclude? Every time the  confidence interval crosses the line,   the vertical line, there is no difference  between active and passive control,   active condition and pa – and control  condition. Every time the confidence interval,   like in this case, it’s entirely on  the right hand side, in this case,   this means that the active treatment is better  than the control. If there was a line completely   here on the left, this will mean that the  control is better than the active treatment. Now, these – each line represents one individual  study. The final effect, the meta-analytic effect,   is represented here, at the bottom, sometimes it’s  represented like a diamond. And, in this case,   you see that the confidence interval is  entirely on the right side of the figure,   so it means that according to parents’ rating,   parent training is better than control, when  we look at the evidence across all studies. However, importantly, if we look at the ratings  provided by Teachers, which we called in this   meta-analysis, probably blind, because, you know,  they are less influenced by the delivery of the   treatment, they’re blinded, they’re not aware  of the allocation, and they cannot guess it,   in that case, unfortunately, it turns out  that parent training is not statistically   different from control. You see here the  final effect, the meta-analytic effect   is on – is actually in the middle, so it  crosses the line, the vertical line. So,   this is an example of pairwise meta-analysis,  with a specificity to consider in terms of ADHD. Let’s move to another type of meta-analysis  which is gaining traction in the field,   it’s becoming more and more popular. This is  called ‘network meta-analysis’. So network   meta-analysis, in a nutshell, is a specific type  of meta-analysis, which allow us to compare,   under certain methodological assumptions  that need to be tested rigorously,   two or more treatments, even if they  have not been compared head-to-head in   the individual trials that are  included in the meta-analysis. This is very important, because usually the  majority of trials we have in child psychiatry,   in general and, also, in other fields, are  active treatment versus placebo or control.   There are few head-to-head trials. However,  what we need as a Clinician is really knowing   how different treatments compare with each  other, because we need to make choices. We   need to select – we need to choose treatment A,  treatment B or treatment C for my patient, so we   would – I really need evidence from head-to-head  trials. In the absence of head-to-head trials,   we can rely on network meta-analysis,  assuming that certain conditions are met. The way we represent the network meta-analysis  resu – is this, so this is called a ‘net plot’.   So, you see each node here is a treatment,  so we may have, in the case of ADHD,   this is a network meta-analysis we published a  few years ago, comparing all the pharmacological   treatments for ADHD, so, methylphenidate,  amphetamines, atomoxetine, and so on and   so forth. When there is a line which connects  the dots, this means that there is at least one   trial comparing directly those two treatments.  So, for instance, there are certainly trials   comparing atomoxetine and methylphenidate. There  are trials comparing methylphenidate and placebo. However, there are no trials, for instance,  comparing atomoxetine and guanfacine,   and we may ask, is atomoxetine better than  guanfacine? Worse? Is there any difference? So,   in the absence of this trial, we look at  the effect of the network meta-analysis. So,   after applying all these complex procedure, we  came with these results, in terms of the efficacy,   in the short-term, of ADHD medications for  ADHD core symptoms. And we have represented   here these medications in red, these are  those which are approved, and, in black,   those that are off-licence,  but that may be used for ADHD. And you see the highest effect size, one, is for  amphetamines. I remind you that effect size is   a measure of the size of the effect, and when  it’s around – from zero to .2, means very low   effect size, so it’s statistically significant,  but clinically very small effect .3 to .56 is a   moderate effect, so cl – there is some change,  and higher than .8 will be high effect size,   so we can really see a tangible, clinically  significant finding, when, you know, patients   and their families are happy for the result of  the medication. So, you see the highest effect   was for amphetamine and the lowest effect was  for atomoxetine, which is still moderate effect. There are also tables reported, I didn’t  report in the slide for sake of simplicity,   but there are tables which are called  ‘net league table’, which allows you to   cross each treatment with each other to  compare, for instance, you may wonder,   is amphetamine better than methylphenidate or  worse? Is amphetamine better than atomoxetine   or worse? And so on and so forth. So, I have  not reported this table, because it’s very big,   but you can consult the papers and, in this  case, we found that amphetamines were better   than methylphenidate, in terms of efficacy, but  methylphenidate had a better tolerability. So,   this is why, for instance, this is in  line with the recommendation of NICE,   to support methylphenidate as the  first-line treatment for ADHD. Right, so, this was another example of  meta-analysis. Another example is the   so-called ‘dose response’ meta-analysis or  network meta-analysis. This is a specific   meta-analysis which can be informative  on the effect of, you know, the dose   in terms of the outcome. So, the question is, for  instance, if I increase the dose of a medication,   do I have a better efficacy pr do I  have a worse tolerability? For instance,   we did a dose response meta-analysis, a network  meta-analysis in this case, in adults with ADHD,   so pharmacological treatment, and we could see  that the more we increase the dose, you see from   0mg up to 85mg, the more we increase the dose,  the more significant reduction in efficacy we had. However, beyond the licence dose,  which is 60mg for methylphenidate,   the gain is not that spectacular. You see, you  have a lot of drop here, significant drop here,   but if you increase beyond 60, you still have some  gain, but this is throughout the group level, it   is possible that some patients may benefit really  from a dose beyond the 60mg, but other will not.   And, also, the more you increase the dose, and  if you go beyond the maximum licenced dose of 60,   you increase the problems with tolerability.  So, once again, this is true at the group level,   so it means that, in general, if you go beyond the  licence dose, you don’t have a spectacular gain in   terms of efficacy, and you start having problems  with tolerability. But this is true at the group   level, some individuals they perfectly tolerate  higher doses of stimulants, and if you have a   signal that there is a partial improvement and it  is well tolerated, you can go off-label pending,   you know, that this is clearly explained  to the patients and their families here. Right, so, finally, I wanted to show you an  example of an umbrella review in the field. So,   as I mentioned earlier, umbrella review is a  specific review which is a collection basically   of meta-analysis, or systematic review in the  field, and it really allows you to have a very   comprehensive view. This is particularly helpful  when different meta-analysis, they focus on the   same population, for instance, ADHD in this  case, but they focus on different outcomes,   so you can pull together everything  and have a very general overview. I wanted to give you an example of an  umbrella review in the field of ADHD,   which was conducted relatively recently, and this  was on actually the safety of medications across   different disorders and so it included, also, ADHD  in children. And this is very interesting graph,   so what they did here was to plot, in terms of the  side effects, so they had a list of all possible   side effects, they calculated, and you see here in  grey the percentage of side effect that have been   assessed in relation to a specific medication. So,  for instance, when we look at the literature on   methylphenidate, and we look at the type of side  effect that have been explored, 32% of that global   list were explored in terms of methylphenidate,  and you see here the percentage for others. And then in black, you see – and so the grey  one are called the ‘adverse events coverage’,   and those in black were those that were  significantly worse with the medication. And so,   of course, the higher the difference between  the black and the grey, the better it is for the   medication. And you see that basically the safety  profile of all medications for ADHD is best – the   best safety profile is for methylphenidate, you  see here, because out of all these side effects,   only a small percentage are significantly worse  due to the medication. So, this umbrella review   highlights that, overall, despite what we read  sometimes in the lay press, and we can, you know,   hear based on a conversation with different  colleagues and other reports in the media,   despite all this scientific evidence, tell us  that methylphenidate is, actually, overall,   quite safe, in terms of tolerability profile.  So, this is just an example of umbrella review,   there are many others that are being  published right now in the field. Right, so, I hope this was helpful  to provide an overview of how we can   use and interpret advanced methods to  synthesise the evidence, to support our   clinical practice, and to inform our clinical  decision-making. Thank you for your attention.