Qualitative Longitudinal Research: From Monochrome to Technicolour

Abstract

Qualitative longitudinal research (QLR) has grown in prominence and popularity since the 2007 themed section on the subject in this journal. This new themed section reflects on how QLR has been mobilised and made sense of in recent times. This article showcases QLR’s distinctive way of knowing and understanding the social world and how it can illuminate the processes through which policy can enhance, or indeed inhibit, the well-being of individuals and groups within society. This state of the art article focuses on QLR as a valuable but tricky approach. It argues that QLR can enhance qualitative research in social policy by fostering sustained policy engagement and development, and that this is especially useful for policy-making in relation to socioeconomic disadvantage. It also explores cross-cutting methodological dimensions pertinent to QLR’s distinct approach, such as its enhanced ethical considerations, which are also integral to research with people at high risk of socioeconomic disadvantage. The article concludes with possible future directions and developments for QLR as a methodology.

Introduction

Qualitative longitudinal research (QLR) has grown in prominence and popularity since the 2007 themed section on the subject in this journal edited by Corden and Millar (2007a, 2007b). Today, there are more QLR studies focused on social policy interventions than at the time Corden and Millar were writing (Dwyer and Patrick, 2021); however, it remains a less common method than its cross-sectional counterpart. On the one hand, this is down to reasons of pragmatism. In situations where a QLR approach would best answer a specific research question, the costs, resources and time involved in meeting with research participants at repeated time points – making it more complex, expensive and time-consuming than cross-sectional approaches – can make it seem prohibitive. It is also more challenging in terms of the ethical dimensions of research. It can create messier (but often richer) relationships between participants and researchers; but, in so doing, also creates heightened risks to the preservation of participant anonymity due to the level of personal detail that repeat research encounters elicit. On the other hand, QLR might not always be the ideal tool for answering certain social policy questions; there are those best suited to cross-sectional approaches, reinforcing the central premise of this themed section that research questions align to the meanings and dynamics of time that aim to capture change and continuity in people’s lived experiences. In spite of these challenges, what is gained from undertaking qualitative research across time can be invaluable. One renowned scholar said that undertaking QLR compared to cross-sectional qualitative research was like going from a black and white to a colour television (Ridge, 2015); it is from this rich image that we take the title of our article. As QLR researchers we believe there are multiple ways in which QLR enhances qualitative methodology. In this article, we focus on some of these ways within the sphere of social policy and explore cross-cutting methodological dimensions pertinent to its distinct approach.

After a brief introduction to the authors’ own engagement and utilisation of QLR in social policy research this article first looks at the utility and possibilities particular to QLR for social policy. Social policy speaks to the areas of people’s lives that are vital to their wellbeing and their human flourishing; yet, sometimes, policy actions or oversights threaten these and increase people’s risk of adversity. QLR allows us to explore the impacts of change on people’s lives and how social policy interacts with these, e.g. having children, experiencing disability, or losing employment. Thus, QLR provides a robust base on which to influence and monitor the impacts of social policy over time. Furthermore, QLR tracks changes at the behavioural level of the individual, and at the institutional level of an administration, which can yield crucial insights and evidence. There are two key changes in policy direction and focus that lend themselves especially well to QLR, and which are specifically pertinent to socio-economic disadvantage; the first is the individualised behavioural accounts of poverty and deprivation that now prevail in the UK, and the second is the increase in powers to devolved and regional administrations within the UK that is leading to increasing policy divergence from Westminster.

The article then reflects on the key methodological challenges and possibilities of QLR, focusing in on the ethical plane. We discuss the need to navigate the enhanced ethical considerations inherent to the method; for example, in ensuring the confidentiality and anonymity of, and maintaining reciprocal relationships across time with research participants. This section of the article also examines some of the developments that have occurred since the first themed section in 2007, focusing in particular on practical issues such as data collection and analysis techniques. It goes on to explore the possibilities inherent within QLR for maximising and extending policymaking engagement; and, within this, the possibilities for adopting participatory approaches. We focus here on the UK context, though we argue our reflections, which are grounded in our own experiences as qualitative longitudinal researchers, have a wider relevance and resonance.

In concluding this article, we share some reflections on future directions for QLR in social policy and in efforts to engage with and influence policymaking. Not only has the interdisciplinary field mobilised QLR to enhance our understanding of social phenomena, but for social policy, its explanatory power lies in helping to explain and evaluate the design and implementation of policy – in particular, the impacts of social policies on people’s lives. With the growth of participation and involvement in policy making, we also highlight the potential benefits to cultivate relationships with policymakers over time, coeval with the QLR studies themselves, and in creating the space and opportunities to develop policy-responsive (and indeed methodological) outputs.

Time as a vehicle and object of study

As authors, each of us collating this themed section is convinced of the special methodological power and richness that is intrinsic to adopting time as both a vehicle and object of study (Henwood and Shirani, 2012). With QLR, time becomes both a methodological tool and a conceptual focus; QLR researchers must engage theoretically and substantively with ideas of time; with changes and transitions in individual lives, and with reworkings and recasting of past, present and future lives. Rich opportunities emerge when looking at and working with time; datasets become stronger and more robust, research relationships are enhanced, and there is increased scope for flexible, iterative policymaking engagement.

Each of us has also adopted qualitative longitudinal methods in a social policy context, examining experiences of teenage pregnancy and motherhood (Wenham, 2016); child and family poverty and inequalities (Treanor, 2018, 2020); and experiences of welfare reform over time (Patrick, 2017). In this article, we draw on our own reflections from these methodological experiences, which we blend with the wider literature on QLR; pulling out the possibilities, but also the challenges characteristic of a qualitative longitudinal approach.

Theorising time

There are two types of time key to QLR projects, the characteristics and conditions that exist at each point in time (synchronic) and the change between these successive points in time (diachronic) (Hermanowicz, 2016). In moving to the diachronic by adding ‘spatial and temporal dimensions to synchronic study’ (Hermanowicz, 2016), longitudinal data allow us to move from a monochromatic outlook to one full of colour.

Time as an object of study in QLR allows the examination of change and of stability. At each research contact, change and the effects of change are recorded and explored with participants. By examining change we can also explore the transitions people make in their lives. Transitions are often seen as a series of discrete events rather than ongoing processes with time as a critical factor. The strength of QLR, as a method for exploring transitions, is that it moves beyond linearity to look forwards and backwards in time simultaneously (Calman et al., 2013). It is iterative in nature and draws on the learning from earlier rounds of data generation to understand how change occurs and the impacts it has (Carduff et al., 2015). QLR also has the potential to enhance participatory methodologies; QL researchers work with their research participants to understand, interpret and contextualise research findings in an iterative process, thereby lending increased analytical power to the research. This provides a more nuanced understanding of the effects of events and circumstances that evolve over time (Carduff et al., 2015). The study of time and of change has implications for QLR research methods: as participants’ lives evolve, research questions can emerge which can have implications for data collection methods (Vogl and Zartler, 2021). Within lifecourse research, time is conceptualised in both absolute and relative terms (Sánchez-Mira and Bernardi, 2021; Bernardi and Sánchez-Mira, 2021).

The fluid nature of QLR and its relationship to time allows us to begin to make sense of that most elusive of concepts: causality (Neale, 2021). Unique to QLR methodology and well exemplified by Neale (2021: 2), QLR can have a distinctive explanatory power in that it has the capacity to illuminate causal processes ‘by following reality in all its windings’. It does so, Neale argues, as it captures the dynamic nature of lives lived and follows ‘processes where they lead and identify their constituent threads; to discern how these threads are connected through time, space and the social fabric; and how they coalesce to create the momentum for change’ (Neale, 2021: 8). Causal processes, Neale posits, are illuminated through QLR and its relationship to time: ‘time does not stand still while causal processes unfold; they are not located in static environments or contexts. They have no temporal fixity, no discrete end point, other than an artificial one that is created as an artefact of the research process and its window of observation’ (Neale, 2021: 9). This is not to say that QLR can, or can uniquely, disentangle causality. The much-contested causal pathways in any research are more than the sum of linear events in time. Rather, while cautious of deducing cause and effect, the tools mobilised to track processes within QLR, allow for a more nuanced understandings of the complex mechanisms through which change can occur. This is especially the case when considering the identification and significance of ‘critical moments’ (Holland and Thomson, 2009) that can be highly consequential for a person’s biography and/or sense of identity.

As well as change, QLR allows for the exploration of stasis and of the protective factors in people’s lives; for example, stable, secure, affordable housing, or the absence thereof, can be understood using QLR and translated into what works well or what we need more of in policy-terms. Bringing in time as both a vehicle and object of study (Henwood and Shirani, 2012), QLR creates possibilities to generate a dynamic and more finely grained picture of responses to social policy interventions, with particular possibilities in exploring how far and whether policy agendas focused on the provision of welfare, including aspects of behavioural change, achieve the desired outcomes (Corden and Millar, 2007a). Given the complexity of people’s lives, this central component of time in QLR methodology affords many advantages to the study of the impacts of social policy in people’s lives (Neale, 2021).

Social policy and QLR

Recent trends in social policy in the UK and internationally encompass policy foci that can be effectively analysed and interrogated with qualitative longitudinal methodologies. This is particularly true of the behavioural turn in social policy, and the growing reliance on welfare conditionality as a silver bullet, that is conceptualised as containing within it the possibility to convert passivity into action; welfare dependence into independence, and unemployment into engagement with paid employment (Lindsay et al., 2018; Watts and Fitzpatrick, 2018). Welfare conditionality relies on policy interventions that nudge, but also seek to compel individuals to change in ways that the state deems desirable, which usually means engaging in the paid, formal labour market. These measures, with their combination of carrots and sticks – best exemplified by forms of work-related conditionality – provide an excellent exemplar for the power of QLR analytical tools. By following people over time, and tracking and contrasting policy intent with lived experiences, we can explore how far and whether the affecting changes in individuals’ lives map onto the policy narrative; and – by so doing – develop a richer and more detailed understanding of the policy’s impact and effectiveness against intended objectives. What is so powerful about QLR is the possibilities within it to track and contrast policy assumptions with happenings; and to tease out both the presence and absence of change over time. This has been done in various research settings, and at various scales, with the UK-based large scale QLR panel study, Welfare Conditionalities: Sanctions, Support and Behaviour Change, which was able to generate a rich and persuasive evidence base documenting the limitations and – in many cases – perverse consequences of welfare conditionality (Dwyer, 2018).

In the aftermath of the 2008 global financial crisis, in diverse national contexts, we witnessed a number of increasingly right-leaning administrations whose ideological approach to social and economic support promulgated individualised and behavioural accounts of poverty and deprivation. As part of deficit reduction strategies, austerity measures were put in place across many wealthy societies (Treanor, 2020). Social policy today exists in a context in which this new politics of austerity is setting the parameters of the debate (Farnsworth and Irving, 2018: 461). In the UK, the past dozen or so years have been a time during which we have witnessed significant welfare reform and cuts to public spending (Beatty and Fothergill, 2013; Dowler and Lambie-Mumford, 2015; Edmiston, 2017). These cuts and reforms in the name of austerity have had socioeconomically and geographically uneven impacts, with the individuals and groups that we would wish to research being the most adversely affected (Gray and Barford, 2018).

The cumulative impacts and effects of austerity since 2008 are best examined through QLR research. The importance of tracking how successive changes to the social security context are experienced over time and the scope to explore the cumulative impact of repeated changes and policy interventions are uniquely apposite to QLR. In Patrick’s work, for example, through following a small number of people over six years, it has been possible to explore how an initially resilient response – to welfare reforms and cuts in support – is superseded by growing financial insecurity and negative mental health impacts (Patrick, 2017). This climate of continual uncertainty and insecurity pervades everyday lives (Patrick, 2017). The rich possibilities with QLR to track these impacts of austerity on individual lives over time is also apparent in work on children’s experiences of persistent poverty by The Children’s Society (2020). A key issue that emerged organically from the process of speaking to children, which was not an initial focus of the research team, was housing insecurity over time. Lead researcher Sorcha Mahony reflected on the value of the methodology in this regard:

…one of the most exciting aspects of doing qualitative, longitudinal research is the opportunity it affords not only to seek answers to predetermined questions, but to explore the unforeseen, to follow leads from participants as they surface and to investigate issues that we did not expect to encounter. (Mahony, 2020)

In social policy, we should always be ready to look up beyond our immediate research preoccupations to focus on the wider context and to understand better the interwoven nature of policy and everyday life. QLR provides the scope to be responsive and adaptive to areas of policy concern, which emerge through the research process. QLR also enables an engagement with how an individual experiences a particular policy intervention across time as part of their own personal journey and as part of their own personal identity. For example, in a QLR study focused on over-50s experiences of a mandated welfare to work intervention (the Work Programme) in the United Kingdom, Neary et al. found that: ‘the use of longitudinal qualitative methods in this study also enabled us to explore the “journey” of candidacy in the intervention. By going beyond the initial impressions, we were able to explore how participants navigated the intervention, and how different interactions affected their perceptions. Also, examining how their own identity and beliefs regarding capacity shaped their understanding of suitability, and interaction with staff influenced these’ (Neary et al., 2020: 11). The QLR study by Wenham (2016) also allowed for the analysis of the interface between how ‘problems’ such as teenage pregnancy are framed within political and policy narratives, with the accounts of those subject to such policy intervention. This not only illustrated the potential for policy to exacerbate stigmatisation, but also how everyday experiences of stigma and shame were then negotiated through time by the young mothers.

These finely grained accounts of individual lives have also been illuminated by the intersection of social policy with other disciplines that have paved the way for the development of QLR, both theoretically and methodologically. For instance, the work of Henderson and colleagues (2007) allowed for a more holistic and dynamic account of young people’s transitions to adulthood. This QLR study followed a diverse group of young people over a ten-year period, allowing for a series of accounts where lives were told backwards (retrospectively), in the present, as well as forwards (prospectively). Through a QLR lens, the authors were able to revisit the past and future with young people as their lives unfolded, allowing rich theoretical contributions to our understanding of youth transitions that have particular policy relevance (i.e. factors that exacerbate social exclusion). This analysis of time, as both a vehicle and object of study, is central to QLR, and is illuminated through a range of interdisciplinary projects that were part of a broader ESRC funded study titled Timescapes (2007-12). This programme of research focused upon a range of empirical investigations covering the lifecourse. Not only did this capture rich descriptive details of individual and family lives over time, but the empirical findings provided explanatory insights into the implementation and evaluation of social policies: for example, the need for social policies to better support intergenerational caring responsibilities as lives unfold over time (see Bornat and Bytheway, 2014; Emmel and Hughes, 2014).

The QLR method: a valuable but tricky approach

Having set out the possibilities that QLR offers social policy analysis, we now move to a focus on the more practical challenges implicit in researching through time, including looking closely at the ethical domain.

Planning a QLR project

One of the key issues in developing a robust QLR project is to plan it as such from the outset. When undertaking cross-sectional qualitative research the researcher usually asks for permission to re-contact the participant as part of the original consent process (with good reason); however, there are practical and ethical reasons why this is not the best way to start a longitudinal study. For one, the time and cost resource implications of QLR means that thought needs to be given to building the reciprocal relationships over time needed to undertake QLR successfully. The major risk to the quality of QL research and to the quality of the sample is participant drop-out. This is a challenge that needs to be managed with care as, although QL researchers make great efforts to keep participants involved, being able to withdraw at any time is a participant right that must be explicitly stated. Reengagement must be by informed and freely given consent. What helps to keep participants willingly engaged in QL research are: the quality of the relationships between researcher and participants, the quality of the contacts and the length of time between contacts. QLR should be a design in and of itself and not an afterthought or an adjunct.

Resource intensity

QLR is resource intensive. It is more expensive than cross-sectional research, takes a lot of project management and needs a lot of time for analysis and for communication with participants. It is not always easy to persuade research funders that the increased costs of longitudinal research are outweighed by the increased possibilities that arise from adopting a longitudinal approach. However, the case can be successfully made, as has been recently demonstrated by the award of a large UKRI grant to a mixed-method study of ‘welfare at a social distance’, which examines welfare in a time of COVID-19 and includes repeat interviews with a number of working-age social security recipients. What is also important to remember is that it can be done at varying scales; with both small-scale and large-scale studies potentially being very effective for generating new insights and understandings (see Dwyer and Patrick, 2021). Indeed, there are costs and benefits for working at either a larger or a smaller scale, and these especially relate to the potential loss of close researcher-researched relationships as the scale of the study grows; which can also make it harder for the analysis effort to fully engage with the richness of a longitudinal, data set. On the other hand, of course, the larger the scale, the more possibilities to be seen to be policy relevant; although this is itself arguably a weakness in how policymakers engage with and understand the possibilities for improved understanding that can emerge from even the smallest-scale QLR projects.

Data generation

Time is of methodological importance in the generation of QLR data; specifically, the optimum number and frequency of contacts with research participants. There is no hard and fast rule; sometimes QLR takes place over months and at other times years. It depends on the research questions and the circumstances or events being researched. In research for Barnardos, Harris et al. (2009) carried out monthly interviews with families and their children to explore the seasonal impacts of poverty. It was critical to be able to see what effects low income was having in the run-up to Christmas and the subsequent impacts in January and February: this research could not have been undertaken with any greater period between interviews. It is important to ensure that the timing of research contact is balanced between the substantive focus of the study and to the practical challenges of leaving too long between contacts as this may weaken research relationships.

In Treanor’s seven-year research with families living in poverty, the element of time made it easier to sustain relationships with research participants, particularly those at greatest risk of persistent poverty, such as those with intersecting inequalities (Treanor, 2020). As socio-economic disadvantage is not evenly distributed, QLR is an ideal vehicle for exploring the interactions of intersecting inequalities on those who live them. The particular sensitivities inherent in research on poverty and intersecting inequalities are attenuated by relationships that are strengthened over time. Yet, despite researchers’ best efforts, some do report difficulties in reengaging with research participants (Vogl and Zartler, 2021). Miller (2015), whose research has always been at the vanguard of QLR methodologies, reflects on the unwillingness of participants to participate again, leading to the researcher feeling that they are ‘stalking’ them. This is a tension that cannot be mitigated completely, especially given participants’ right to withdraw; however, it can be ameliorated by the participants knowing when, for how long, and with what frequency they will be contacted.

Also critical in the data generation process of QLR is the possibility to adapt the methodological tools to respond to and interrogate an evolving policy context. By researching through time, it is possible to respond to changes as they happen, rather than having a singular, static opportunity to engage with the policy context at the point of time of the research encounter. This can inevitably heighten the policy relevance and the timeliness of the research conducted in ways that can be beneficial for broader processes of impact and influence on the policymaking process. Related to this (and to QLR’s analytical process) is the extent to which QLR can proceed iteratively, with analytical insight from one interview wave being used to inform the design of the next. For example, where there are new explanations and understandings that emerge around responses to a particular policy intervention, these can then be explored further by adding particular focus on this topic in subsequent interviews. This approach was beneficial in Patrick’s (2017) work on experiences of welfare reform where an early wave of interviews showed how many participants were struggling with a change in payment frequency in their benefit. This had not been an original focus of the study, but given the importance participants were attaching to it, Patrick added a question specifically on payment frequency to the next interview wave. This generated rich data, and data that were especially relevant, given the concurrent move to monthly payments under the UK’s introduction of Universal Credit.

Data generation over a five-year period also allowed Wenham (2016) to illustrate the importance of viewing teenage pregnancy and motherhood as fluid and dynamic, moving away from static snapshots that often depict young mothers as struggling (or ‘failing’) at one point in time, to more nuanced accounts that reflect their journeys into motherhood and the changes that occur through time. Again, this reinforces the importance of seeing time as both an object and vehicle of study. Whilst qualitative enquiry is renowned for giving us access to the rich details of social phenomena, QLR allows for greater purchase on how perspectives, attitudes, experiences, and circumstances change through time. The value of this for social policy is profound; capturing change and continuity through prospective and retrospective accounts can potentially illuminate the critical junctures for effective policy intervention.

Analytical techniques

In QLR, the meaning and significance of data emerges as researchers interpret the data at each individual time point, often in collaboration with the participant, and then again over time when time becomes a factor in the analysis. This is as important as ‘specific research emphases, questions, and themes may change over time’ (Hermanowicz, 2016: 499). This means, on a practical level, that the analysis of QLR data is carried out cross-sectionally and longitudinally. The data are analysed at each time point and also longitudinally to explore change over time to establish the net effects of change (or stability) in people’s lives. This allows the researcher to examine change caused by previous events and circumstances but also to explore situations that may have been prevented due to pre-emptive action at an earlier stage. The researcher is constantly looking backwards and forwards in time. This requires a great deal of thought and skill in qualitative data analysis. Calman et al. (2013: 2) advise to ensure not to present data as cross sectional descriptions but also to focus on the effects of time, of the processes of cause, consequences and of change. There are specialist software packages (for example, Framework matrices) to assist with the analytical effort, which is a demanding but rewarding one, through which researchers can seek to blend the cross-sectional (synchronic) and longitudinal (diachronic) analyses; as well as exploring the interplay between the two (Neale, 2021). The utility of framework matrices is especially apparent in large scale panel qualitative longitudinal studies; where there is a real analytical challenge tied to managing large data sets with several points of data generation. Framework matrices, as a method of analysis, can allow researchers to see cases, according to themes, over time, and so ensure that the analysis that proceeds is grounded both in the temporal and in the richness of the accounts produced through the methodology (see Dwyer and Patrick, 2021). This article has focused upon QLR studies that predominantly use tools such as qualitative interviewing and thematic analysis, but there are a vast array of qualitative tools, techniques and methods of analysis that are useful for a QLR approach. Generating data that engage with temporal processes can involve the crafting of ethnographical, visual (such as photo elicitation), participatory tools (life maps, diaries) and documentary data. These modes of longitudinal enquiry often complement each other and can be used in combination to provide a rich and detailed data set of different temporalities.

The ethics of researching through time

In QLR studies, through repeated contact, the ethical plane becomes especially important, which warrants dedicated attention and reflection: ethics need to be understood as situated and flexible, as pro-active and re-active. QLR also brings increased practical and personal risks; but it is the ethical plane that, over time, compounds risk and is therefore the more complex (Neale and Hanna, 2012; Neale, 2013). QLR places the researcher in a privileged position: it allows them special knowledge and, in so doing, the researcher bears particular responsibility (Treanor, 2018). With the privilege of repeated access to participants, comes a range of challenges and responsibilities as well as opportunities. For example, the enhanced ethical considerations involved in visiting the same people repeatedly means that special attention needs to be paid to the depth of personal and potentially disclosive data generated due to the relationships between researchers and research participants strengthening over time.

Many of the topics we seek to understand in social policy are highly personal, often stigmatised (see Wenham, 2016), and can be a source of great stress for research participants. In meeting someone once, you can discover much, but a crucial qualification is that you will never know what you did not find out, you will never know whether that person is responding candidly or naturally, and you will never know what they chose not to reveal. When you undertake QLR these constraints are somewhat attenuated; you build a relationship with someone, you come to understand their ways of communicating, you gain their trust, you learn and you understand so much more. For example, in Patrick’s research into experiences of welfare reform it was only in the second and third interviews that some participants shared substance addictions and (in two cases) shared how they relied on ‘survival theft’ by shoplifting for essentials for them and their children when finances were especially tight (Patrick, 2017).

One particular ethical consideration inherent in working with the same research participants over time is how and when to finish the research and end the research relationship. These considerations of closure need to be properly thought through, built into the project and, where possible, mitigated. Treanor (2018) emphasises having a defined end date to the research, identifying when is the right time to withdraw, and understanding that research participants deserve to live their lives free of researcher scrutiny. Treanor calls this closure of relationships an ‘exit strategy’; whereas Calman et al. (2013) call it ‘escaping the field’ and give practical advice on how to do this. In their study, participants were prepared for the longitudinal element by receiving clear information that they were going to be interviewed four times over the course of a year (Calman et al., 2013). Furthermore, researchers prepared participants for the last interview both prior to and at the point of the last interview and had an informal post-research debriefing session with them (Calman et al., 2013). This facilitates the closure of relationships in a planned, managed and ethical fashion.

In keeping with all qualitative research, it is important to recognise and conceptualise informed consent as an iterative process and to ensure to secure informed consent throughout the research. With QLR, however, there is the additional onus to secure informed consent afresh at each research encounter, during engagement, disengagement and re-engagement, and across a longer research period. Tied to this is a linked challenge of ensuring that participants have opportunities to withdraw from studies and making sure that this is not compromised by efforts to sustain engagement (or in more positivistic language – reduce attrition). Qualitative longitudinal researchers regularly adopt a range of sophisticated strategies to sustain contact over time: engaging with people on social media; securing a number of contact details; and making contact between interviews to keep the relationship alive; and to ensure that they are informed of any changes in contact details. While these strategies are all valid, they must be finely balanced with providing the opportunity for participants to disengage and to choose to withdraw their participation in the relevant study. What is vital, here, too are the possibilities for participants to disengage, but then also re-engage at a later stage; when their situation changes and/or they feel that they are ready to take part in the study once more. Researchers also need to be alive to the danger that participants may feel under obligation to continue or may wish not to disappoint the researcher by ceasing to continue to engage in the research. This could be particularly pertinent for smaller QLR studies, where researcher and research participant relationships are potentially more intense.

There is also a heightened difficulty in ensuring confidentiality and anonymity over time as people’s lives, and events in their lives, can be quite unique and so hard to document while maintaining anonymity. This needs robust management and a willingness to strip back the data sufficiently prior to publishing in order that the identity of the participant is not disclosed. In some cases, anonymity may not be the objective, but this is less common in social policy research, especially when it is undertaken with heavily stigmatised populations, which is often the case.

Particularly important, and something that we have explored elsewhere (see Patrick, 2012a, 2012b) is the challenges of ethically maintaining and managing a reciprocal relationship with research participants that you know a great deal about and whom you have seen year after year after year. Treanor (2018) notes, the researcher ought to be cognisant of the fact that the data generated are much more revealing than would be the case with cross-sectional qualitative research. The relationship between participant and researcher inevitably and rightly changes and is changed by the research process and the very act of repeated encounters. In all our work, these encounters are underpinned by an ethics of reciprocity and of care. QLR more broadly tends to follow an ethics of care sensibility, which is important for the reasons identified in the article; however, this does not avoid there being ethical dilemmas that emerge from the field; and that require both proactive and reactive strategies and responses. Set against this context, it can be both important but at the same time incredibly difficult, for the researcher to maintain distance.

Other ethical issues that are amplified by conducting research with the same participants over time are: the risks of intrusion into people’s lives, the risks of distortion of experience due to repeated contact and personal involvement, and the risks of dependency (Calman et al., 2013: 3). When undertaking QLR in any discipline the researcher has to be aware of the effects they have on the research participants. When undertaking QLR in social policy research, the researcher also has to be cognisant of how being part of the research may affect the participants’ lives, including materially changing it, thus impacting on the research itself. An example from Treanor’s QLR on the impacts of welfare reform on families living in poverty was when a particular research participant was asked about income, including benefit receipt, and it became clear that they were not receiving their full entitlement. As a result of the research highlighting the existence of certain benefits, the participant went on to successfully apply for them, a direct consequence of participating in the research. This had the effect of increasing their incomes and improving their material conditions. If QLR were to be approached from a positivist standpoint, this would be seen to be interfering in a research participant’s life and thus affecting the research. However, in conducting QLR, there is a moral imperative, especially in the study of poverty and deprivation, to ensure the research participant is in receipt of all state support to which they are entitled. This is ethically the right thing to do even if, when you next visit the research participant, their circumstances an improved due to their involvement in the research. This demonstrates how QLR is underpinned by an ethics of reciprocity and of care. Calman et al. (2013: 5) describe this as a ‘complex, grey’ area in QLR: which ‘may become harder to separate, or manage ethically, empathy as a human being and a wish to help people who are suffering, with the role of a researcher when relationships deepen over time’.

With QLR, one has to consider the potential negative and positive effects on the participants of being part of a research project over time. QLR usually requires the research participant to engage in a high degree of reflexivity in their lives, circumstances, experiences and choices. This means that what emerges from the interviews must do so in a safe and trusted environment. Research participants often report that they value the opportunity to think and reflect, that they find it therapeutic and, in Treanor’s work, have said that the researcher is the only person they can speak to and that they look forward to next seeing her. How the participants view the researcher’s role is important and ought to be cause for ethical reflection, thought and a willingness to adjust research approaches in the light of this.

Researchers can also be affected by their role even if they are well-trained, have good support protocols in place and have planned well: QLR can be emotionally challenging (Calman et al., 2013). QLR depends on the strength of relationships over time and the researcher may come to care deeply about their research participants. Hearing about distressing situations is difficult and this can have an emotional effect on the researcher. Some QLR researchers report the ‘blurred boundaries’ and taking ‘emotional work’ home with them (Calman et al., 2013). In the many, complex ethical considerations of QLR it is important to be highly reflexive in one’s practice. Warin (2011: 805) explains that ‘ethics and reflexivity are closely intertwined’ at every point in QLR and proposes a method of ‘ethical mindfulness’.

ethical mindfulness is an alertness or heightened sensitivity to understanding the relational aspects of the research process: an interdependent awareness of how I, as a researcher, am influencing my research participants’ perceptions and a simultaneous and interdependent awareness of how they are influencing me. This representation of the meaning of ethical mindfulness appears to describe reflexivity equally well, revealing the interdependency between the two concepts (Warin, 2011: 809)

In summary, the ethical plane in QLR is complex and requires sensitive and careful navigation. The ethical dilemmas that qualitative researchers encounter are heightened due to the repeated contact between the researcher(s) and their participants; while new challenges, specific to the multiple points of contact, also require negotiation. These are especially notable around the risk to anonymity of repeated contact, and so detailed disclosure of individual lives; and of the changing relationship between the researcher and participant over time. It is important too to have ethical sensitivities towards the researchers in the team who may become more emotionally involved in the research as they form closer relationships with research participants. In each of our cases, we have found that having a strong and robust ethical framework – underpinned by ideas of care and reciprocity – has been invaluable in navigating this difficult terrain; while it has also been important to recognise the inevitability of encountering ethical tensions; which then need to be re-actively responded to in a sensitive and timely way.

Policymaking engagement and QLR

Thomson and McLeod (2015) emphasise that QLR can capture ‘the accumulation of small shocks into temporally varied pathways and complexities of lived lives’ and that this provides unique insights into particularly intractable policy problems. Social policy research conducted within a QLR framework has relevance to policymakers on several fronts. Firstly, QLR provides rich opportunities to respond iteratively to a shifting policy context and to provide findings of relevance timeously to policymakers. Compared to cross-sectional approaches, which limit the research moment to one interaction, there are repeated opportunities to explore the policy field in question, which can create rich and often much needed opportunities to respond to changes in the relevant policy context. For example, in Patrick’s study of lived experiences of welfare reform, an ongoing longitudinal study that began in 2011, a small cohort of individuals who are variously impacted by recent benefit changes was established to monitor change. As new changes are announced and rolled out – such as Universal Credit – there are then opportunities to return to this cohort and find out how they are faring; and how the changes are impacting upon them and their lives. This new data and insight can then be speedily and efficiently fed back to policymakers.

At the same time, the iterative nature of the QLR research process encourages an approach that enables and promotes several points of dissemination and the production of interim outputs. This has recently been done very effectively in a large-scale QLR project of couples’ experiences of Universal Credit; with an interim output on first-wave findings being disseminated and contributing to the conclusions of the House of Lords Economics Committee on Universal Credit. While adopting a QLR approach enhances the possibility of responding to a shifting policy context and creating multiple points of policy dissemination; there remains a challenge (not specific to QLR) of engaging UK policymakers to facilitate a translation of research evidence into policymaking processes.

In this time of increasingly austere and punitive approaches to social policy in the UK, there has been an increased transfer of responsibility for social policies to the devolved nations of the UK. It is within the locus of these new, young administrations that longitudinal and biographical accounts across time have had most impact on increasingly-devolved policy areas. In Scotland, for example, there has been legislative and policy change, in part influenced by longitudinal projects, such as a QLR project undertaken by the Child Poverty Action Group that began in 2013, called the Early Warning System, in which Treanor (co-editor of this themed section) collaborated. The influence of researchers and campaigners occurred in real-time as the research evidence emerged.

For QLR to have policy impact there has to be a willingness on the part of politicians and policymakers to engage with researchers, those with lived experience, and a desire to effect evidence-based change. There remain questions about how far policymakers currently want to use evidence (Monaghan and Ingold, 2019) and for what purpose. But these questions arise regardless of the nature of the research conducted and so are not specific to QLR (Dwyer and Patrick, 2021).

As research develops and strengthens across time so too can the relationships and engagement with policymakers. This is incredibly powerful and affords unique opportunities for influence. When you have these key players as an attentive audience they will be keen to hear how the story unfolds. By demonstrating how change in social policy affects families’ lives across time there is a possibility to have real-time impacts on policy developments. The key is to build relationships with policymakers. This is often easier for those in the third sector who often have the skills and dedicated resources for campaigning and influencing rather than for carrying out research or analysing data. For this reason, involving a third sector partner in QLR research can create a powerful partnership which can help influence social policy.

Conclusions: future directions for QLR

Bringing the temporal centre stage methodologically and conceptually has immense and still often untapped value for social policy analysis. It creates an incredibly rich analytical terrain; and one from which it is possible to look back and forwards at both the absence and presence of change. This is particularly pertinent for an engagement with social policy interventions orientated at generating behavioural change, as highlighted in the first themed section on this topic back in 2007 (Corden and Millar, 2007a). In this article, we have sought to think through the particular possibilities but also the potential challenges in working through and across time; including on reflections from our own research practice and the wider literature.

In concluding, we would like to pull out three key areas of potential further development for the methodology; each of which has a very tangible relevance for social policy. First, there is considerable and still untapped scope for aggregating data from qualitative longitudinal studies on overlapping themes; and blending and building the resultant evidence base (Wright and Patrick, 2019). This has real power to help us to access the ‘shared typical’ of experiences of particular policy processes, interventions or policy problems (e.g. welfare conditionality; street homelessness), while also building the case for the relevance and even the scope to generalise from qualitative research (2019). Linked to this, there is also the possibility in building in more secondary analysis of qualitative longitudinal datasets, which is becoming increasingly possible because of the move towards archiving data for re-use by researchers and other core stakeholders (see Hughes and Tarrant, 2019; Neale, 2019; Davidson et al., 2021). There is a strong ethical case for making more use of existing qualitative data sets; and this is especially true of longitudinal data sets, due to the increased depth of the available data; and the increased demands made on participants to access their accounts and experiences, not just once but potentially several times. Qualitative secondary analysis can also be utilised to inform the design of future empirical qualitative longitudinal studies; refining their focus; and ensuring that new work is effectively orientated and grounded in the existing evidence base (see, for example, Tarrant, 2017).

Secondly, with the potential for QLR to maximise and extend policymaking engagement, we highlight the possibilities for adopting more participatory approaches. We propose that there is scope to join together the recent growth in interest in participatory methodologies that tap into the expertise that comes with experience; especially around studies that touch on social security, homelessness, poverty and disability (Beresford, 2016; Wright and Patrick, 2019); with the possibilities inherent with researching through and across time. Both approaches succeed or fail on their abilities to sustain research relationships over time; and in both, at least as concerns social policy research, there is frequently an interest in influencing and contributing to policy debates. In a recent participatory study on experiences of Universal Credit in Northern Ireland, Patrick and Simpson (2020) worked in partnership with a small group of claimants of the benefit to develop their experiences and create recommendations for change. This was facilitated through regular workshops; which continued across a year. As the project did not have a specific temporal focus or dimension, longitudinal data was not explicitly collected; but this was an arguably missed opportunity to bring time in; and to better understand experiences across the life of the project, which would have only strengthened the resultant findings. It would also have allowed a better understanding of how the very process of being involved in participatory projects can have beneficial outcomes for all parties; in ways that perhaps strengthen the case for incorporating processes of co-design and co-production in the policymaking process.

Thirdly, we would like to reiterate the very real opportunities that QLR provides for real-time policymaking engagement; and a policy engagement that builds over time as research continues; in ways that can potentially boost and deepen research impact. In this regard, there are possibilities to develop and strengthen relationships with policymakers over time; just as research relationships are developed with participants; and in ways that can create chains of trust with the potential to cut through into real impact and even processes of social change. This is evidenced across a number of QLR studies, and is explored further as part of this themed section by Scullion et al. (2021), who document the ways and extent to which their research into the experiences of veterans claiming social security was more impactful because of the relationships with the Department for Work and Pensions being built up and developed over time, and alongside the research itself.

Each of these strands are possibilities where we think there is potential to build upon and develop our use of QLR in social policy research. Overall, though, it is important to re-state the central argument running through this article: although QLR is time intensive and fraught with ethical challenges, it is worth this additional labour – for the richness, and the finely grained insight it can produce. In a policymaking context, the value of this level of detail, and the close attention to processes (and absences) of change it enables can hardly be over-stated. Researchers who switch to the technicolour view that it enables rarely want to return back to the previous black and white, which is itself more than testament to the strength of the method.

The use of longitudinal qualitative methods in this study also enabled us to explore the “journey” of candidacy in the intervention. By going beyond the initial impressions, we were able to explore how participants navigated the intervention, and how different interactions affected their perceptions. Also, examining how their own identity and beliefs regarding capacity shaped their understanding of suitability, and interaction with staff influenced these.