Triangulating evidence from the GALENOS living systematic review on trace amine-associated receptor 1 (TAAR1) agonists in psychosis

Abstract

Background

Trace amine-associated receptor 1 (TAAR1) agonists offer a new approach, but there is uncertainty regarding their effects, exact mechanism of action and potential role in treating psychosis.

Aims

To evaluate the available evidence on TAAR1 agonists in psychosis, using triangulation of the output of living systematic reviews (LSRs) of animal and human studies, and provide recommendations for future research prioritisation.

Method

This study is part of GALENOS (Global Alliance for Living Evidence on aNxiety, depressiOn and pSychosis). In the triangulation process, a multidisciplinary group of experts, including those with lived experience, met and appraised the first co-produced living systematic reviews from GALENOS, on TAAR1 agonists.

Results

The animal data suggested a potential antipsychotic effect, as TAAR1 agonists reduced locomotor activity induced by pro-psychotic drug treatment. Human studies showed few differences for ulotaront and ralmitaront compared with placebo in improving overall symptoms in adults with acute schizophrenia (four studies, n = 1291 participants, standardised mean difference (SMD) 0.15, 95% CI −0.05 to 0.34). Large placebo responses were seen in ulotaront phase three trials. Ralmitaront was less efficacious than risperidone (one study, n = 156 participants, SMD = −0.53, 95% CI −0.86 to −0.20). The side-effect profile of TAAR1 agonists was favourable compared with existing antipsychotics. Priorities for future studies included (a) using different animal models of psychosis with greater translational validity; (b) animal and human studies with wider outcomes including cognitive and affective symptoms and (c) mechanistic studies and investigations of other potential applications, such as adjunctive treatments and long-term outcomes. Recommendations for future iterations of the LSRs included (a) meta-analysis of individual human participant data, (b) including studies that used different methodologies and (c) assessing other disorders and symptoms.

Conclusions

This co-produced, international triangulation examined the available evidence and developed recommendations for future research and clinical applications for TAAR1 agonists in psychosis. Broader challenges included difficulties in assessing the risk of bias, reproducibility, translation and interpretability of animal models to clinical outcomes, and a lack of individual and clinical characteristics in the human data. The research will inform a separate, independent prioritisation process, led by lived experience experts, to prioritise directions for future research.

Despite extensive endeavours in recent decades, there has been only limited progress in identifying novel therapies with new mechanisms of action for people with mental disorders. In the area of psychosis, new clinical paradigms have been introduced such as ‘early intervention’ and longer-term management with the development of better-tolerated depot medications, but nearly all pharmacotherapy still relies on the modulation of dopaminergic systems within the brain. Drug discovery centred on non-dopaminergic medications has been unsuccessful, with most drugs that have shown promise in preclinical animal studies subsequently failing in clinical trials.^1,2 Therefore, as part of the broader aim of novel drug discovery, any assessment of a promising new agent or new drug class in this area (non-dopaminergic medications) needs to assess both the animal and human sources of evidence for efficacy, adverse events and mechanisms of action.

Trace amine-associated receptor 1 agonists

Trace amine-associated receptor 1 (TAAR1) agonists are a novel approach and mechanism for treating psychosis,³ but their mechanism of action in psychosis is not fully defined. It is thought that TAAR1 agonism may have efficacy by regulating presynaptic dopamine signalling.³ Currently, two TAAR1 agonists, ulotaront (SEP-363856, TAAR1 agonist and serotonin 5-HT_1A receptor partial agonist) and ralmitaront (RO6889450, TAAR1 partial agonist) have been investigated, and additional compounds (e.g. RO5256390, ZH8651) are undergoing preclinical development.^3–5 However, recent clinical trials have had inconclusive findings despite showing promise in preclinical studies, and there is ongoing uncertainty regarding the effects and potential role of TAAR1 agonists in the treatment of psychosis, as well as the differences between individual TAAR1 agonists, the differences in effects in animals and humans, and their exact underlying mechanism of action.³ As the volume of data on TAAR1 agonism is rapidly increasing, and additional compounds are undergoing preclinical development, we are carrying out a living systematic review (LSR) to incorporate all the present and new evidence about TAAR1 agonists that will be produced in the next years, in human and animal studies.

Challenges in evidence synthesis for novel treatments

There are significant challenges in assessing the evidence and developing recommendations for novel treatments. One challenge is to ensure that summaries of evidence and recommendations are up to date and keep pace with the expanding body of evidence (including the evidence from experimental studies and early phase trials), and a helpful approach to this is to have LSRs.⁶ These are syntheses of evidence that are updated regularly, as needed, to incorporate new evidence as it becomes available. LSRs are particularly relevant in areas where the research evidence is emerging rapidly, is uncertain or has the potential to change policy or practice,⁶ and as such, they are suited to many fields of psychiatry and mental health.

Assessing the certainty of evidence is another challenge, as replicability remains a significant problem⁷ (see Box 1) and biases are often repeated even with adequate replication. One way of addressing this problem is the use of triangulation^8,9 (see Box 1). This is a process of evidence synthesis where different sources of evidence, and therefore different sources of bias, are considered together. This enables the integration of a wider range of approaches that otherwise would be rejected because of their level of bias, allowing a broader assessment, which can be particularly helpful where study data are early in development.

Box 1 Assessing the certainty of evidence for novel treatments and triangulation

Challenges with assessing the certainty of evidence

(a) Methods such as the Grading of Recommendations Assessment, Development and Evaluations (GRADE) approach¹⁰ can be used to assess the robustness and reliability of research findings, but replicability of the evidence remains a significant problem.⁷

(b) Strategies to increase replicability might improve this, but simple replication of studies in itself may not yield robust conclusions.

(c) This is because replication of a study may also replicate the inherent bias within the study, and so there is a strong argument for a different approach, such as triangulation.

Triangulation

(a) Triangulation is a process of evidence synthesis where sources of evidence with different types of bias are considered together.^8,9

(b) The process explicitly acknowledges that systematic errors (or biases) are present in each study approach, but these biases are likely to be unrelated when different study approaches are assessed.¹¹

(c) Potential biases in one study design would not be expected to significantly influence estimates in a different study design or method.

(d) Therefore, if the results of several different approaches all point to the same outcome, this will strengthen confidence in the findings.⁸

(e) This is particularly the case if the approaches have potential biases that would favour findings in opposite directions.⁸

(f) The triangulation method also enables the integration of a wider range of approaches which otherwise would be rejected because of their level of bias.

(g) For example, animal studies are often limited by the lack of animal models available and their translational validity and by issues of replicability and potential biases. Clinical data from early phase studies are often sparse and dose-response studies are not easy to conduct.

(h) Triangulating animal and human evidence allows a broader assessment of the available approaches and can be particularly helpful where study data are early in development.

Objectives

We aimed to evaluate the available evidence on TAAR1 agonists in psychosis and develop recommendations for future research and its prioritisation. We used a multidisciplinary co-production approach, including lived experience experts,¹² to appraise the evidence from human and animal studies by using triangulation.

Method

The GALENOS research programme

This study is part of a larger research programme in mental health, the Global Alliance for Living Evidence on aNxiety, depressiOn and pSychosis (GALENOS; https://galenos.org.uk/). GALENOS is a multidisciplinary international collaboration where evidence in specific areas of mental health is extracted and synthesised. Online open-access LSRs and data-sets are developed that facilitate the translation of this evidence into recommendations for research which may lead to clinical applications.¹²

In line with the wider programme of GALENOS, the methodology used in this study included co-production between clinicians, researchers and experts by experience throughout the process, and use of triangulation to assess the evidence from a variety of sources, to develop recommendations for research prioritisation, future investment, practice and methodologies. This is the first time we have used this methodological approach, so we have described the process we followed in more detail. This study reports the findings of this process in assessing the evidence from the first GALENOS LSRs, on TAAR1 agonists in psychosis.

Choice of topic

Psychotic disorders rank among the top 20 causes of disability worldwide,¹³ but despite this clear global burden of disease, the development of new treatments with better tolerability and efficacy has been slow.² There is an urgent need for these, especially in low- and middle-income countries.

Current approaches for antipsychotic medications focus mainly on D2 receptor antagonism. Although existing antipsychotic drugs can be effective for some symptoms (such as hallucinations, delusions and agitation) and can prevent relapse,^14,15 there are high treatment non-response rates.¹⁶ In addition, current treatments are often limited in their ability to improve other key symptoms, such as lack of motivation or cognitive impairment, which negatively affect activities of daily life.¹⁷ These medications can also have multiple side-effects, including weight gain and movement disorders.¹⁴ In recent years, new approaches and mechanisms of action have been investigated, and the topic of TAAR1 agonists for psychosis was chosen as a priority by the multidisciplinary group within GALENOS.

Collecting the evidence from animal and human studies

The detailed background, methods and results of the LSRs for animal and human studies are already described elsewhere.¹⁸ Data were extracted from identified studies in multiple electronic databases up to 28 August 2023 for the animal studies and 17 November 2023 for the human studies. The protocol was registered with PROSPERO (International Prospective Register of Systematic Reviews) (identifier CRD42023451628) and Open Science Framework¹⁹ (OSF) (identifier https://doi.org/10.17605/OSF.IO/86Z2P), and published in Wellcome Open Research.²⁰ The LSRs, developed with multiple international stakeholders and co-produced with experts by experience (hereafter the ‘GALENOS expert group’), aimed to understand if TAAR1 agonists are effective at reducing the symptoms of psychosis, what adverse events they might be associated with and their potential mechanisms of action, including data from clinical (human) and preclinical (animal) studies. Although preclinical studies as a whole can include a variety of research approaches, for the LSR on preclinical data relevant to TAAR1 agonists in psychosis we considered only data from animal studies.

Setting up the triangulation panel

In line with the consensus development panel approach^21,22 used by the National Institutes of Health and World Health Organization,^23,24 we asked a separate panel of multidisciplinary expert participants (the ‘triangulation panel’) to assess the different sources of evidence. This approach helps to mitigate any potential biases from the GALENOS expert group.^25,26 The triangulation panel composition was gender-balanced, and professional backgrounds and expertise included clinical psychiatry, clinical studies, preclinical studies, research methodology, evidence synthesis, statistical analysis and lived experience. The chair of the group (J.P.T.H.) provided expertise in evidence synthesis methodology, LSRs and triangulation. The triangulation panel was global (including Australia, France, Germany, India, The Netherlands, Switzerland, Greece, Italy, the UK, the USA and Zimbabwe).

Co-production

Equal partnerships with people with lived experience of mental health conditions are central to GALENOS, to increase the relevance of findings at each stage.²⁷ At the governance level, there is a Global Lived Experience Advisory Board (GLEAB), an international and demographically diverse group that co-designs each stage of the programme and oversees the engagement strategy in collaboration with and supported by MQ Mental Health Research (https://www.mqmentalhealth.org/home/). For this study, co-production was central to the methodology and used the Guidance for Reporting Involvement of Patients and the Public (GRIPP-2)²⁸ short form as a guide (see Supplementary Appendix available at https://doi.org/10.1192/bjp.2024.237). Two experts in lived experience (N.G. and T.K.) formed part of the triangulation panel who assessed the output of the TAAR1 LSR and participated in the triangulation of the sources of evidence. They were supported at the meeting by an MQ Mental Health Research team member with a background in co-production approaches in mental disorders (L.M.). To prepare for the triangulation meeting, as well as review all the materials provided to the other triangulation panel members, the two experts in lived experience also met with the GALENOS Director (A. Cipriani) to understand the scientific details of the studies assessed and to ask questions to clarify any areas where technical terms or jargon might prevent understanding of the results.

Triangulation meeting

A meeting of the triangulation panel and the GALENOS expert group to discuss and triangulate the evidence from the TAAR1 agonists LSR was convened on 15 February 2024 in London, UK. This was a hybrid meeting, with some participants attending in person and others remotely.

The process of triangulation aimed to answer three key issues:

1. (a) To consider whether the sources of evidence from animal and human studies were consistent in showing similar effects, taking into account the direction and magnitude of any potential biases within the studies.

2. (b) To agree on recommendations for future work and research prioritisation in the area of TAAR1 agonists for the treatment of psychosis, given the sources of evidence to date.

3. (c) To provide the GALENOS team with feedback to be considered in future iterations of this review to guide further research prioritisation as more data become available.

Relevant material was circulated to the panel in advance of the triangulation meeting, including a video with slides of the key methodology, summary of evidence tables for the animal and human data¹⁹ (https://osf.io/84wfm, https://osf.io/wpd78), and the results of the two LSRs.¹⁸ Summary of evidence tables are used to provide the main findings of a review to allow an assessment of the magnitude of any effect and its certainty.²⁹ In this study, the findings from the LSRs were presented for each outcome for the different sources of evidence (i.e. animal and human studies) in the rows, and the different domains relevant to the confidence of the evidence in the columns.¹⁸ The structure of the summary of evidence tables and the domains considered were predefined in the protocol¹⁹ and included the source of the evidence, a summary of the association, internal and external validity assessments, and reporting biases.

During the meeting, the lead reviewers for the human data (S.S.) and for the animal studies (M.M.) presented the main findings to the whole meeting and answered factual questions raised by the panel. The GALENOS expert group provided clarification on the data where needed, but did not participate in the discussion among the triangulation panel. The meeting followed a preset agenda (see Supplementary Appendix) with allocated time for presentation and discussion of the animal and human data, triangulation and recommendations. Minutes were taken and the meeting was recorded with the permission of all attendees.

Reflexivity statement

GALENOS is fully supported by the Wellcome Trust. The meeting was convened by the GALENOS expert group, which also included three members from the Wellcome Trust (N.B., J.M. and K.D.). The triangulation panel was chosen to represent a balance of professional backgrounds and clinical and scientific knowledge, lived experience and gender, but the group was not systematically selected. We recognise that each member's contribution may come from multiple areas of experience including professional and personal, and that each may bring a different set of strengths, but also possible preconceptions and biases. As with any group endeavour, we acknowledge that the shared knowledge and experiences of the GALENOS expert group and the panel may have had an impact on the interpretation of the data.

Results

Following the structure of the agenda, the panel first considered the animal and human sources of evidence in turn using the summary of evidence tables,¹⁸ and after each presentation they had the opportunity to seek clarification on any points (for examples, see Supplementary Appendix). The panel then discussed the data in more detail, including the evidence on efficacy, outcome measures and safety in the animal and human studies, followed by the process of triangulation.

The first part of triangulation assessed whether the sources of evidence from animal and human studies showed similar effects, taking into account the direction of any potential biases. The output is contained in Table 1.

Table 1 Triangulation question 1: Is the evidence from animal and human studies consistent in showing similar effects, taking into account the direction and magnitude of any potential biases within the studies?

The triangulation panel then considered recommendations for future work and research prioritisation in this area (TAAR1 agonists), given the evidence to date. The output is contained in Table 2.

Table 2 Triangulation question 2: What are the recommendations for future work and research prioritisation in this area (trace amine-associated receptor 1 (TAAR1) agonists and psychosis), given the evidence to date?

Recommendations for the next iteration of the LSR are outlined in Table 3, and include (a) carry out a meta-analysis of individual human participant data; (b) actively seek and add any study design beyond randomised controlled trials (e.g. single-arm studies, or non-randomised controlled trials) and (c) expand the remit of the LSRs beyond the diagnosis of psychosis to other disorders, such as depressive disorder or bipolar disorder, and beyond the previously studied symptoms.

Table 3 Triangulation question 3: What areas need to be considered in future iterations of living systematic reviews on trace amine-associated receptor 1 agonists to guide further research prioritisation as more data become available?

During the process of triangulation, several wider issues were noted in assessing animal data in general, including the challenges of reproducibility and possible publication bias, and the translational relevance of the animal models used for mental disorders, including psychosis. In the clinical data, the wider issue of accurate measurement for symptoms in studies of psychosis was also raised: although standard ratings (such as the Positive and Negative Syndrome Scale)³⁵ may be useful, measures of other specific symptoms (such as cognition, anxiety, low mood, motivation, sedation, functioning and quality of life) or wider objective health outcomes (such as hospital admission, additional medication use) may also be relevant when considering a novel antipsychotic mechanism.

Discussion

Principal results

Using triangulation, this study assessed the output of LSRs investigating TAAR1 agonists in psychosis. The methodology allowed the global multidisciplinary group to assess a wider range of evidence across both animal and human studies. Triangulating the sources of evidence provided clear recommendations about TAAR1 agonists and psychosis for the next iterations of the LSRs and for future research in this area.

Areas of challenge in the animal and human data

In addition to providing recommendations related to TAAR1 agonists in psychosis, the triangulation process also raised wider issues around the quality of evidence, particularly from animal data. Assessing animal and human data together is critical because animal research provides fundamental information to our understanding of the underlying biological mechanisms that underpin disorders and their treatment. However, to make an assessment, the evidence must be as robust and reliable as possible, with measurable and transparent sources of bias.³⁶ Although the assessment of bias in human studies is well defined,³⁷ the risk of bias (and adequate reporting to assess the risk of bias) in animal studies is not. Reproducibility in animal experiments has proved to be challenging, with variability in experimental design, conduct, analysis and reporting.³⁸ Selection bias, performance bias, detection bias and attrition bias are all common issues.³⁹ Several tools are available to support researchers in planning and reporting studies, as are risk-of-bias tools for assessing preclinical animal studies and their inclusion in systematic reviews. However, even when these are used, publication bias or selective reporting is common in the animal literature.³⁹ For example, one study found only around 26% of the animals used in experiments were reported in subsequent publications.⁴⁰ New, open research practices including preregistration of preclinical animal studies, are needed to help with improving experimental design and reporting,³⁶ but systemic change will also be needed within the research community.³⁸

In addition, triangulation highlighted issues around the translation and interpretability of animal models to clinical outcomes. The majority of studies reviewed in the animal data LSR regarding TAAR1 agonism used the reduction of locomotor activity induced by a ‘pro-psychotic drug’ as the main indicator of efficacy, and not a more general translational or disease model. This approach favours identifying those drugs with similar mechanisms of action to existing antidopaminergic antipsychotics, but it may be less useful in identifying new drugs acting through novel mechanisms or targeting different pathways, or symptom domains other than positive symptoms.

There are also some challenges with the clinical data. The human studies together showed a small reduction of psychotic symptoms compared with placebo, but this was only in one study. They had a favourable tolerability profile, but without clear evidence of superiority over existing antipsychotics on the measures used. There are a number of reasons why the data may have been variable. Individual and clinical characteristics, such as specific symptoms, chronicity of illness, baseline severity of symptoms, age and gender, which may be predictors or mediators of response to medications, were not well described in the individual studies, as they generally reported aggregate data only. This is particularly important in the assessment of a new treatment approach to psychosis where only subsets of patients with specific patterns of illness, symptoms or other characteristics may benefit from the new mechanism of action. For example, because of the adverse side-effect profile, existing antidopaminergic medications may be problematic, particularly in the early stages of psychosis,⁴¹ but there is evidence that delay in treatment may worsen overall outcomes.^42,43 Given their favourable side-effect profile, TAAR1 agonists might therefore have a particular role early in the course of symptom development, or even in at-risk states where current antipsychotic use is debated,^44,45 but more detailed, head-to-head comparison data are needed. In addition, up to 40% of patients with schizophrenia are treatment-resistant to currently available first-line antipsychotics,¹⁶ with only 40–50% of these responding to clozapine, the single pharmacologic agent approved for patients with treatment-resistant schizophrenia.⁴⁶ There are also currently no medications approved for negative symptoms or cognitive dysfunction associated with schizophrenia, with little randomised controlled data for these symptom domains.⁴⁷ Novel agents such as TAAR1 agonists may therefore have a particular role in some of these clinical presentations, but more detailed individual data are needed.

Placebo effects were also important: in two of the ulotaront phase three studies there were large placebo responses,¹⁸ which may have resulted in an underestimate of the average treatment effect. More generally, evaluation of the quality and validity of animal and human trials for novel agents and of the effects of placebo response present further challenges.⁴⁸ Finally, it is also possible that a longer illness duration and prolonged treatment with antidopaminergic agents may change receptor physiology and treatment response in ways that a short washout period cannot neutralise. This means that the effects of novel mechanism of action agents might behave differently depending on whether a person (or animal) has been pretreated for a long time with postsynaptic dopamine blocking agents. Factors that are related to illness stage also need to be observed in drug development. For example, these were partially considered in the clinical development programme of ulotaront, where – at least in the phase 2B study, which showed favourable results – only patients aged 18–40 years and with no more than two prior hospital admissions for acute exacerbation of psychosis were studied versus a placebo.³¹

Reflections on the process of triangulation

The current study describes the process of international multidisciplinary collaboration to use a triangulation methodology in assessing the evidence from an LSR in a specific area of mental disorder. As this is the first group of animal and human LSRs within GALENOS, we also assessed aspects of our methodology that performed well, and areas where we plan to implement changes and improve subsequent iterations of our LSRs and triangulation efforts.

The results show that meaningful discussion and recommendations for future research and clinical applications can be produced within a co-production international framework. The next step will be a separate and independent prioritisation process, led by MQ Mental Health Research (https://www.mqmentalhealth.org/home/) and lived experience experts, to prioritise the directions for future research based on the findings from the LSRs and the triangulation meeting. The process of triangulation involved experts across different academic and clinical fields, and the co-production with experts by experience (with lived experience in mental health) provided a range of perspectives. Although appraisal of the current data and input from interdisciplinary academic expertise are critical, the benefits of co-production with people with lived experience are being increasingly recognised.⁴⁹ This is particularly important in the appraisal of evidence, to highlight aspects of lived experiences that may not have been considered and alternative interpretations of the evidence.

Potential limitations and future directions

Although co-production with experts by experience runs throughout GALENOS, there were some areas where the process of co-production could be improved. For example, the technical and scientific language of the evidence contained within the LSRs is complex. To mitigate against this potential problem and enable an equal voice, the experts by experience had interactive meetings before the triangulation. However, the discussion within the triangulation meeting involved interrogating the evidence and raising scientific and methodological questions, which may have inhibited active contribution in some areas from the experts by experience. Although the whole meeting group was asked to avoid acronyms and scientific terminology, there were areas where this was unavoidable. In future iterations, the meeting will also provide a glossary of terms with lay definitions to be used before and during the meeting to facilitate equal understanding, and will include a lived experience co-chair.

The triangulation process relies on a variety of evidence and, ideally, results from multiple methodologies or disciplines. Key elements are that, where possible, sources of evidence should be triangulated from published and unpublished sources, as well as across methodologies.^9,11 In this first iteration of the TAAR1 agonists LSRs, the studies included randomised controlled trials, but, because of the novel investigation of these agents and small number of very recent studies, uncontrolled experimental studies were included only for the mechanistic insights in human studies. In addition, although pharmaceutical companies were contacted for additional human studies and missing data, and clinical trial registries were searched for unpublished studies, not all of the data were available for the LSRs. This was particularly so because TAAR1 agonists are new agents and so data from recent studies have not yet been analysed. Unpublished animal data were also searched for, but this was challenging as preregistration for animal studies is not yet an established practice. Thus, although these approaches ensured a timely development of the first iteration of the LSRs, it is possible that they may have missed other sources of evidence. However, the strategies used were predefined in the first version of the protocol, which was preregistered at OSF,¹⁹ with a real-time record of updates, and this prospective preregistration of the protocol is a key element in the triangulation process.⁵⁰ In addition, this was the first version of the LSRs, and so future iterations of the review will be updated to include searches for other types of study.

In conclusion, in this first iteration of a dual LSR using a global multidisciplinary group to triangulate key aspects of the sources of animal and human evidence, we provide clear recommendations about TAAR1 agonists and psychosis for future research. The next key step in the GALENOS process is to co-produce research priorities in this field, which can be used by funding agencies to develop mental health interventions that are equitable, impactful and viable at scale.