In Defence of Ordinary Help: Estimating the effect of Early Help/Family Support Spending on Children in Need Rates in England using ALT-SR

Abstract

Randomised controlled trials are often inappropriate for many forms of preventative children’s services; as such, observational studies using administrative data can be valuable for evidence-based policymaking. However, estimates of effectiveness can be confounded by differences in thresholds of intervention and national policies that exert pressure on local trends. This study adjusted for these factors using methods developed in clinical psychology to control for individual traits and developmental trajectories, Autoregressive Latent Trajectory Models with Structured Residuals, to analyse the relationship between local authority preventative spending and Children in Need (CIN) rates in England. Higher spending was associated with significant decreases in CIN rates between 2010/11 and 2014/15, but not from 2014/15 onwards. In the first half of the decade, 1% increases in expenditure were associated with between 0.07% and 0.157% decreases in CIN rates. Based on average local authority spending cuts, this translates to an additional 13,000 to 16,500 children and young people put or kept at risk of developmental or health impairments nationally for each year between 2011 and 2015. These findings highlight the potential of early help/family support policies and concerns around how their effectiveness has changed consequent to prolonged austerity and a deliberate policy focus on ‘what works’.

Introduction

Under section 17 of the Children Act 1989, local authorities in England have a duty to provide services that ‘safeguard and promote the welfare of children within their area who are in need’. Children who are at risk of having their health or development impaired without the provision of additional support are recorded in administrative data as ‘Children in Need’ (CIN). Support for these children and their parents is often delivered through various forms of ‘early help’, ‘early intervention’, or ‘family support’ that typically aim to prevent existing or potential problems escalating to safeguarding concerns (Frost et al., 2015). Greater reinvestment in early help and family support services, which have been diminished since 2010 (Webb and Bywaters, 2018) has been touted as a solution to the problem of growing rates of children in care (NCB, 2017; Featherstone et al., 2019).

Analyses by the National Audit Office (NAO, 2016, 2019) and Ofsted (2016) reported there was no relationship between expenditure on preventative children’s services and their quality or rates of child protection interventions. The implication of these findings has been that investment alone is inadequate for improving outcomes. In his 2016/17 annual report, the Chief Inspector of Ofsted Michael Wilshaw wrote:

“We now know that: inadequacy is not a function of size, deprivation or funding, but of the quality of leadership and management.”

Ofsted, 2016: 5, quoted in Lavalette, 2019: 28

A significant amount of evidence from Randomised Controlled Trials (RCTs) and meta-analyses generally concludes that family support services have beneficial impacts on children’s social, emotional, and educational development, and on family functioning and parenting skills (Hutchings et al., 2017; Axford et al., 2015; Allen, 2011; MacMillan et al., 2009; Melhuish et al., 2008; Dunst et al., 2007; Dagenais et al., 2004; Layzer et al., 2001) – however, as indicated by the NAO’s analyses, these may not necessarily translate into decreased state intervention rates. Commissioning more effective early intervention services based on high-quality evaluations has been an aim of UK government throughout the 2010s, outlined in the Allen Report (Allen, 2011) and reinforced in the establishment of the ‘What Works Network’ (Department for Education, 2013). Despite the Allen Report’s ambition of only guiding new funding, and not threatening existing programmes, the ‘quality’ of evidence – as measured by how closely evaluations align to a framework that positions experimental and quasi-experimental methods as the ‘gold-standard’ – has become a significant policy driver in deciding what services are funded under austerity.

’What works?’ and the defence of ordinary help

Critics of what could be called the ‘what works’ paradigm have noted that many early help and family support services do not conform well to, or are ethically inappropriate for, RCT evaluations, particularly where these services are delivered in community settings or utilise less manualised or structured forms of support (Stewart-Brown et al., 2011; Stewart-Brown, 2012; White et al., 2014). As White et al. (2014: 83) state simply: “the more ordinary and relatively cheap the help, the less likely it is to yield to experimental methods”. Thoburn et al. (2013) highlight several features of ‘ordinary help’, including that it is flexible, adaptive, and sensitive to the context of multiple needs; it is relationship-based, and includes help with pragmatic factors that can strain parenting. Jack and Gill (2010) highlight that it includes a range of activities that create informal social support structures, assuaging the mistrust and power imbalance between parents and social workers.

These services are often highly tailored to the needs of the population they support, sometimes explicitly because they originate from community self-organisation, which further complicates the appropriateness and generalisability of RCTs. Though there is no way to establish the size of reductions to the provision of services with experimental evidence against those without, the case of children’s centres in England may serve as one example due to their area-based implementation, universality, and provision of multiple varied services. Smith et al. (2018) report that between 2009 and 2017 more than 30 per cent of registered children’s centres had closed and 55 per cent of local authorities reported reduced services, with increased focus on specialist provision for complex needs and reduced universal support (see also: Hood et al., 2020b).

This is particularly acute when a distinction between ‘early intervention’ and ‘early help’ is made. While the two terms are used interchangeably, one pattern, but by no means the absolute rule in England, is that ‘early help’ more generally refers to the need for intervention or support provided early in the life of a problem, whereas ‘early intervention’ often stresses the need for intervention on perceived risks early in the life of a child, as well as early in the life of a problem (White et al., 2014; Featherstone et al., 2019). Much experimental evidence comes from evaluations of interventions with very young children, especially in the field of neuroscience (Wastell and White, 2017; Featherstone et al., 2019). Services for adolescents often focus on relationship-building with trusted adults and practical, long-term support, usually embedded in physical spaces like youth centres. This places services for older children in a particularly precarious position. The YMCA (2020) report that youth services funding has been cut by more than 70 per cent between 2010/11 and 2018/19, with the closure of 1 in 6 youth centres between 2012 and 2019.

Policy rationales for commissioning public services increasingly coalesce around causal evidence based on medical models of science (Wastell and White, 2017), economic justification (Featherstone et al., 2019; Maron, 2021), and potential for financialisation (Wiggan, 2018; Jones, 2018). This can drive out community solutions that often lack the capital to demonstrate such outcomes. Of the Early Intervention Foundation’s eight early help programmes that it rates as having the highest quality of evidence, seven were developed in the USA and one in Australia. Of these, five require license and training fees to be paid to private enterprises, two require paid training only, and one can only be offered exclusively through a UK non-governmental organisation (author’s analysis of EIF Guidebook, June 2020).

‘The trouble with thresholds’: variation and rationing of ‘need’ in administrative data

Establishing causal evidence through non-trial methods is complicated by several dynamics of administrative data. While statuses like ‘in Need’ are defined in legislation, it is the responsibility of local authority children’s services to operationalise these definitions. This results in the ascription of ‘need’ differing between children’s services. In England, each local authority employs differing thresholds for ‘Child in Need’ status, which can be affected by demand for services (Broadhurst et al., 2010), type of risk (Devaney et al., 2012; Hayes and Spratt, 2012), rationing in response to budget constraints (Devaney et al., 2012; Devaney, 2019), and arrangement of referral systems (White et al., 2015). Many drivers of changing thresholds, and therefore rates, are associated with national, rather than local, policy effects over time.

This introduces difficulties in drawing general conclusions from observational studies. Most forms of analysis, including those used by the NAO, rely on observing relationships between cases – for example, whether higher CIN rates are associated with lower expenditure. This relies on the assumption that ‘in Need’ means the same thing in every local authority. In practice, the most high-level forms of intervention become the focus of analysis – namely, rates of children in care or child protection plans – because these interventions are bound by more universal thresholds such as family court rulings. This sets an unrealistic expectation on early help services when their focus is to address children’s needs under section 17 of the Children Act 1989. The effects of early help on care rates or child protection plans may be small or incremental and therefore difficult to evidence in terms of ‘statistical significance’ when the number of children’s services is finite and relatively small (Stewart-Brown, 2011). Additional observational approaches that avoid the pitfalls of administrative children’s services data are required to fairly represent the value of services that do not conform well to experimental methods.

The need to address these confounding factors in longitudinal analysis is not unique to the case of children’s services, and techniques have been developed in clinical statistics to separate ‘traits’ (analogous to differential operationalisations of ‘need’), ‘trajectories’ (analogous to change over time in rates associated with supra-local policies) and ‘within-unit dynamics’ (analogous to associated changes in need and service provision independent of local authority level trends and interpretations of legislation). This study uses Autoregressive Latent Trajectory Models with Structured Residuals (ALT-SR) (Curran et al., 2014; Mund and Nestler, 2019) that adjust for such confounding factors to examine the lagged effect of early help and family support expenditure in local authorities in England on CIN rates. The article contrasts estimates from an ALT-SR model to three other approaches.

Methods

An ALT-SR model represents a combination of two routinely used models in structural equation modelling (SEM): the Cross-lagged Panel Model (CLPM) and the Latent Growth Model (LGM) or Latent Curve Model (LCM) (Curran et al., 2014; Mund and Nestler, 2019). The purpose of a CLPM is to estimate the effects of multiple variables on one another over a series of sequential time points, while controlling for immediately prior values (autoregression) from each unit (individual or case measured over time) under observation. The purpose of Latent Growth Models (LGM) is to model the developmental trajectory of a variable over time (Duncan and Duncan, 2009). This is achieved using latent variables. In a simple example, latent variables may represent an intercept point and a linear slope; these variables can further have variance parameters estimated, which describe how the intercept and slope differ for each individual unit. This variance can further be regressed on or correlated with other variables, including latent growth factors associated with a second developmental trajectory to identify ‘parallel processes’.

The autoregressive components of a CLPM fail to account for stable between-unit factors, which has been shown to bias the estimates for cross-lagged relationships and confound the extent to which they represent within-unit dynamics between two variables over time (Curran et al., 2014; Hamaker et al., 2015; Berry and Willoughby, 2017). There is a need to separate ‘stable differences between units’ from ‘temporal, within-unit dynamics’ (Mulder and Hamaker, 2020: 1). The need for a further decomposition of trajectories was highlighted by Curran et al. (2014). They argue that because many variables exhibit some constant developmental process over time this should not be treated as a within-unit dynamic. This separation is important because, as Curran et al. (2014) explain:

“[M]any contemporary theories … posit complex reciprocal relations between multiple constructs at both within-person and between-person levels of influence, and these relations may vary in magnitude or form across time or over group. However, many traditional statistical models commonly used in practice are restricted to the estimation of between-person relations … and thus may at times provide less than optimal empirical tests of our theoretically-derived research hypotheses.”

Curran et al. (2014: 3)

This logic is extended to local/national government, as developmental trends that are associated with supra-local developments – for example, austerity, should not be considered part of within-local authority dynamics and should be appropriately decomposed into a separate, between-unit part of the model. This can be achieved through the combination of a CLPM and an LGM. Figure 1 shows a basic ALT-SR as a path diagram and highlights the decomposition of variance to the between-unit and within-unit parts of the model.

FIGURE 1. The ALT-SR Model as a combination of a CLPM and LGM

Visual Explanation

Figure 2 provides a visual example illustrating the removal of variance associated with latent trends and the impact of this on the interpretation of cross-lagged relationships within a single case. Plot (a) represents data for expenditure and Children in Need rates at nine intervals between 2011 and 2019, with linear trend lines for each variable over time. Before adjusting for trends, the value of expenditure at 2012 represents a simple decrease, and this decrease is associated with a decrease in CIN rate in the following year.

FIGURE 2. A visual explanation of the effects of removing variance associated with case intercepts and slopes on the interpretation of lagged effects

Plot (b) shows how adjusting the values within the local authority for the larger trend over time changes the interpretation of values to be their fluctuation from an established trajectory. The residuals from this adjustment then form the cross-lagged component of the ALT-SR model to estimate within-unit effects of deviations in spending on CIN rates, and vice versa, above and below the general trajectory. Plot (c) labels one side of these cross-lagged relationships with curved arrows. One consequence of this is that positive expenditure residuals are now consistently associated with negative CIN rate residuals in the following year, better representing within-unit dynamics.

Model Building and Selection

While Curran et al. (2014) prescribe no specific model building strategy, they demonstrate building an ALT-SR model by first establishing an appropriate function for growth over time in each variable, then testing autocorrelation and unidirectional cross-lagged associations between variables with both fixed and freely estimated effects over time. Model selection is achieved through the use of comparative fit. In this study, the robust Comparative Fit Index (CFI), robust Tucker Lewis Index (TLI), the Standardised Root Mean Square Residual (SRMR), the Akaike Information Criterion (AIC), and the Bayesian Information Criterion (BIC) were used to gauge and compare model fit. A summary of these statistics can be found in Kenny (2020). Model selection and the model building process is reported in full in appendix A; the final ALT-SR model included random intercepts, random slopes, and a fixed quadratic growth curve for expenditure with no residual autocorrelation; a random intercepts, random slopes growth curve for CIN rate with residual autocorrelation; and freely estimated cross-lags between both variables.

Assessing Model Fit

Cut-off criteria established by Hu and Bentler (1999) are frequently used to determine whether models represent ‘good’ fits to data; these are set at values close to or greater than 0.95 for CFI/TLI and less than 0.08 for SRMR. The AIC and BIC have no cut-off criteria, and operate as relative indicators for competing models, with the BIC applying a larger penalty for complexity. Smaller values of AIC and BIC indicate better fit. Strict cut-offs for model rejection have been cautioned against due to their behaviour with different sample sizes and degrees of model complexity and misspecification (Marsh et al., 2004, 2005; Nylund et al., 2007; Niemand and Mai, 2018; Shi et al., 2019). A simulation study by Shi et al. (2019) found that the CFI and TLI of correctly specified models falls as sample size decreases. They argue that “A sample of N = 200 observations only provides a reasonable estimate for CFI and TLI when [the number of observed variables is less than] 30” (Shi et al., 2018: 330).

Niemand and Mai (2018) recommend the use of flexible cut-off values, but these cannot yet be readily calculated for models other than Confirmatory Factor Analysis (CFA). As an illustrative example, Niemand and Mai’s (2018) cut-off values for CFI, TLI, and SRMR for a CFA with a sample size and number of observed/latent variables equal to that of this study are 0.927, 0.913, and 0.066 respectively. This adjustment makes the requirements for good fit under CFI and TLI indices less stringent, but requirements under the SRMR more stringent, due to its bias towards small samples. While this is not necessarily a suitable adjustment for an ALT-SR model, it provides an approximate indication of the extent to which claims of good fit below Hu and Bentler’s cut-offs may be inappropriate.

Comparison to other lagged effects models

A number of approaches have confronted the same challenges addressed by ALT-SR. In addition to a comparison with a CLPM, this study also shows differences in estimation between the ALT-SR model and a Fixed Effects Linear Panel Model (FE-LPM), as well as a Maximum Likelihood Dynamic Panel Model (ML-DPM) (Allison et al., 2017). For each of the models where such specification is possible, both fixed and free lagged effects were estimated. For the FE-LPM, models were estimated with and without a lagged dependent variable (Allison, 2015). Each of these models offered only a partial separation of between-unit relationships from within-unit dynamics. The FE-LPM model allowed for the inclusion of differing case-level intercepts and linear trajectories in the dependent variable through the inclusion of local authority fixed-effects and interaction effects, but lagged effects could then only be estimated as fixed over time. The ML-DPM allowed adjustment for differing intercepts of CIN rate and their covariance with expenditure at each time point, but not for trajectories.

Estimation

Model estimation for structural equation models was performed using the lavaan (0.6-7) package in R version 3.6 (Rosseel, 2012; R Core Team, 2019). Estimation of FE-LPM was performed using the fixest (0.7.1) package (Bergé, 2018). Structural equation models were first estimated using a maximum likelihood estimator with robust standard errors, then bias-corrected 95 per cent confidence intervals were calculated from 10,000 bootstrap samples per model. For FE-LPM, clustered standard errors were used to calculate 95 per cent confidence intervals. As there is no external package for estimating ML-DPMs in R these were specified manually in lavaan using code from Allison et al. (2017). ALT-SR models included time-adjacent residual covariance parameters to control for the effects of unobserved time-varying covariates (Grilli and Varriale, 2014; Isiordia and Ferrer, 2018); missing data was handled using Full Information Maximum Likelihood (Little et al., 2014). All data and analysis code are published in an online repository: https://github.com/cjrwebb/cin-spend-rv

Data

Data on local authority Children in Need rates are from England’s Children in Need Census (Department for Education, 2019) and data on expenditure is taken from Section 251 local government spending returns (Department for Education, 2020). Non-safeguarding, non-children looked after (Non-SG/CLA) expenditure was the total expenditure spent on the following categories: Sure Start and early years; family support services; services for young people; youth justice; and ‘other’ children’s and families’ services. As such, this largely captures early help and family support services as opposed to child protection social work activities or services for children in care. Precise early help and family support expenditure categories are not possible to derive due to inconsistency in reporting between local authorities and changes to spending categorisation over time (Webb and Bywaters, 2018). Per capita estimations of spending were calculated using ONS population estimates for people aged 0-17 by local authority. Expenditure and CIN rates were transformed to their natural log values, meaning cross-lagged regression coefficients represent percentage changes in the one variable for a one per cent increase in the other. Descriptive statistics for all untransformed variables are provided in table 1.

Table 1. Descriptive Statistics for Expenditure, Children in Need rates, and IMD Score (pre-transformation)

Indices of Multiple Deprivation (IMD) scores were sourced from the Ministry of Housing, Communities and Local Government as a measure of local deprivation (MHCLG, 2015). The IMD is not comparable between years, and is calculated every five years. IMD scores are based on income deprivation (22.5%); employment deprivation (22.5%); education, skills and training deprivation (13.5%); health deprivation and disability (13.5%); crime (9.3%); barriers to housing and services (9.3%); and living environment deprivation (9.3%). IMD scores were standardised for all analyses. A one-standard deviation increase in IMD score is approximately equal to an increase of 10 per cent of the population with equivalised household incomes that are less than 60 per cent of the median national income (Webb et al., 2020).

Data corresponds to the 152 upper-tier local authorities that children’s services are organised into. Two local authorities were excluded: the City of London and the Isles of Scilly. Both of these local authorities serve very small populations and are not generally representative of typical local authorities in the country. Further, Haringey was excluded from the analysis due to it being a very significant Children in Need rate outlier in 2010/11, having almost twice as high CIN rates in that year than the second highest local authority. This was likely a consequence of the serious case review into the death of Peter Connolly in the years immediately prior (Jones, 2014). Two very affluent local authorities had values for expenditure and Children in Need rates in some years that were noticeably low. Model coefficients did not change significantly if these outliers were removed.

Results and Interpretation

Comparison to CLPM, FE-LPM and ML-DPM

Model fit, cross-lagged regression estimates, and latent variable means and correlations are shown alongside 95 per cent bias-corrected bootstrap confidence intervals in table 2. The ALT-SR model differs in model fit, point estimates, size of confidence intervals, and inference of significance on their basis from all other models. Estimates from each alternative model were not consistent with one another but the ways that the models differed was consistent with what would be expected given their differential treatment of unit intercepts and trends.

Table 2. Comparison of ALT-SR, CLPM, FE-PLM, and ML-DPM

Bolded estimates represent Bias Corrected Confidence Intervals that do not cross zero.

Estimates for the effects of expenditure in a given year on CIN rates trended upwards of ALT-SR estimates in the freely estimated lagged effects CLPM, in the fixed CLPM, in both FE-LPMs, and in the fixed ML-DPM. In particular, the significant negative estimates for the effect of lagged expenditure on CIN rates in the ALT-SR models were estimated to be significant in a positive direction in most cases for the CLPM, where there is no within/between decomposition of variance (Hamaker et al., 2015). In the FE-LPMs, where adjustments are made for individual intercepts and slopes for CIN rate only, the lagged effect of expenditure was negative but not statistically significant regardless of whether an autoregressive parameter was included ( ${\beta _{{y_t}{x_{t - 1}}}}$ = −0.04[−0.112, 0.032]). The ML-DPMs estimated lagged effects that were more positive than the ALT-SR before 2014/15-2015/16 and more negative after. This matches what would be expected given how the ML-DPM adjusts for random intercepts in the dependent variable and their association with values of the independent variable but does not consider growth, therefore likely overestimating effects at earlier time points and underestimating later time points when trends are negative.

This comparison shows that ALT-SR models may be valuable for estimating cross-lagged effects in studies where there is theorised to be a large influence from wider trends that exist either at the level of local government or from national policies. There are many such cases in social policy, where national policymaking, or even global trends, can confound variations in data at the local level, where policy implementation can often diverge and become a valuable source of natural variation. In such cases, it may be undesirable for these effects to be present in the analysis when within-unit relationships, such as the effectiveness of local services, are of interest. This may be especially true if such effects are hypothesised to influence both predictor and outcome variables in an analysis and may not be possible to fully control for using other approaches.

Interpretation of Lagged Effects

Model output for key parameters of ALT-SR models with and without the inclusion of IMD score are presented in table 3. Full model output, which includes variance and covariance estimates as well as factor loadings, is supplied in appendix B. A 1 per cent increase in expenditure in 2010/11 was associated with a −0.159 per cent decrease in CIN rate in 2011/12 ( ${\beta _{{y_2}{x_1}}}$ = −0.159 [−0.248, −0.068]). In the following pair of years, a 1 per cent increase in expenditure was associated with a −0.131 per cent decrease in CIN rate ( ${\beta _{{y_3}{x_2}}}$ = −0.131 [−0.208, −0.050]), and for 2012/13–2013/14 a 1 per cent increase in expenditure was associated with a −0.097 per cent decrease in CIN rate ( ${\beta _{{y_4}{x_3}}}$ = −0.097 [−0.165, −0.024]). The effects of years following this were not significant according to bootstrapped confidence intervals, though the value for the regression of CIN rates in 2014/15 on expenditure in 2013/14 was significant according to a conventional p-value ( ${\beta _{{y_5}{x_4}}}$ = −0.070 [−0.135, 0.002], p=0.040), and therefore may reflect a meaningful effect size through to 2014/15.

Table 3. Model Output for Final ALT-SR Models

As the interpretation of coefficients for logged variables is relative it is necessary to scale coefficients to assess whether efficacy of service expenditure has decreased. Equivalent percentage increases required to match a 1 per cent increase in expenditure in 2010/11 were calculated for years from 2011/12 onwards to scale coefficients. Absolute changes in CIN rate were calculated using these scaled coefficients based on average CIN rates for each year. Scaled coefficients and average absolute change is presented in table 4. This shows that the efficacy of expenditure on reducing CIN rates fell absolutely as well as relatively throughout the decade from a −0.551 per 10,000 change in CIN for a £4.82 increase in spending per child to a −0.254 per 10,000 reduction for the same amount spent in 2013/14.

Table 4. Scaled Coefficients for Lagged Expenditure Effects

Table 5 presents the size of the effects based on average reductions in expenditure from 2010/11, as recent reductions in expenditure far exceed £4.82 per child. This provides a more policy-relevant estimate of the impact of real changes in funding over the decade, holding trajectories over time constant. It is important to state when using ALT-SR models that this is not necessarily an indication of how ‘austerity-free’ CIN rates may have looked, because national trends likely condition rationing within LAs, but an insight into the extent to which cuts to preventative services have contributed to rates of Children in Need if the thresholds and other trends that emerged under austerity are held constant.

Table 5. Expected Effects for Average Reductions in Expenditure under Austerity

Between 2010/11 and 2011/12, Non-SG/CLA expenditure on services fell by around £98.42 per child in an average local authority, a 20.4 per cent reduction. The expected increase in CIN rate within local authorities the following year is estimated to be around 3.65 per cent or 12.8 CIN per 10,000. For 2012/13, 2013/14 and 2014/15, the three years where the effect of lagged expenditure on CIN rates was significant or close to significant, the expected increase in CIN rates was approximately 14.5, 13.9, and 11.5 per 10,000 respectively. This represents between 3.16 to 4.04 per cent increases each year. Because the effects beyond 2015 were not statistically significant, we would not expect to see a consistent change in CIN rates.

This is important considering the apparent emerging reciprocal lagged effect of CIN rates on expenditure that is significant from 2016/17. A 1 per cent increase in CIN rate in 2016/17 was associated with a 0.226 per cent increase in expenditure in 2017/18 ( ${\beta _{{x_8}{y_7}}}$ = 0.226 [0.037, 0.436]). This increased to around 0.344 per cent in 2018/19 ( ${\beta _{{x_9}{y_8}}}$ = 0.344 [0.103, 0.609]). This may indicate that a greater number of local authorities are responding to increases in CIN rates by increasing their expenditure on early help and family support services. There is a concern that services are now reinvesting into a preventative system that has become ineffectual.

Trends in Spend and CIN Trajectories Over Time

Previous research has identified that Non-SG/CLA expenditure on services reduced dramatically in the first half of the decade (Webb and Bywaters, 2018). Because the model used logged values and quadratic components, which can be difficult to interpret, predicted trends for each local authority, and for high, low, and average levels of deprivation, have been back-transformed and are plotted in figure 3 to examine whether this trend appears to have continued throughout the rest of the decade.

FIGURE 3. Predicted trajectories over time from latent growth variables in ALT-SR model

Holding within-local authority dynamics constant, expenditure has continued to decrease throughout the 2010s in England. These decreases were larger for more deprived local authorities and smaller for less deprived local authorities ( ${\beta _{{X_\beta }z}}$ = −0.018 [−0.027, −0.009]). Higher deprivation was associated with higher expenditure intercepts ( ${\beta _{{X_\alpha }z}}$ = 0.234 [0.195, 0.278]). The combination of these two patterns means that variation associated with deprivation has reduced over time, reflecting a more equal but perhaps less equitable distribution of resources from central government.

Trends in CIN rates have also been negative over the decade. Rates were generally higher in high deprivation local authorities and lower in low deprivation local authorities in 2010/11 ( ${\beta _{{y_\alpha }z}}$ = 0.191 [0.157, 0.227]), and have fallen faster in high deprivation local authorities than in low deprivation local authorities over the decade ( ${\beta _{{y_\beta }z}}$ = −0.007 [−0.013, −0.0005]). This may reflect decreasing need at a national level over time, but based on existing evidence may more likely reflect higher CIN thresholds related to rationing of services (Devaney, 2019, Hood et al., 2020a, 2020b, Smith et al., 2018). Lastly, the ALT-SR model tests for correlations between trends that are not already attributable to IMD score. Local authorities that had higher expenditure intercepts also had significantly larger ‘reductions’ in CIN rates over time ( ${\rho _{{Y_\beta }{X_\alpha }}}$ = −0.442 [−0.677, −0.140]). Given the interpretation above, this may imply that local authorities with higher expenditure in 2010/11 were less able to retain their thresholds for offering early help services to families.

Discussion

The legacy of the Allen Report for Children in Need: ten years on

An ambition of the Allen Report was to improve the effectiveness of support provided to children and families through greater use of early interventions that have high-quality evaluations. This ambition has influenced UK policy, including in the funding of the ‘What Works for Children’s Social Care’ centre. The new evidence presented here suggests that increased effectiveness from investment in ‘what has been shown to work’ over the past ten years has not materialised, at least not in terms of reduced rates of children at risk of health or developmental impairments for equivalent levels of spending. Concerningly, effectiveness appears to have declined.

This decline in effectiveness may reflect an unintended consequence of this form of evidence-based policymaking under austerity. Severe cuts have fallen on preventative services (Webb and Bywaters, 2018), and a patchwork of evidence suggests that reductions in provision have been exacted on services that do not meet the ‘gold-standard’ of evidence including children’s centres and youth centres (Smith et al., 2018, YMCA, 2020). There is evidence that the support retained has created a shift away from universal, open-access provision and ‘ordinary help’ and towards provision of targeted programmes for multiple complex needs (Smith et al., 2018, Hood et al., 2020b). In years where these services were more plentiful the effectiveness of local authority spending to prevent risks to child health and development appears to have been greater. The fears of researchers at the time, that this epistemological paradigm might diminish the effectiveness of children’s services, appear to have been well-founded (Stewart-Brown et al., 2011; Stewart-Brown, 2012; White et al., 2014).

Many trees, few forests: the need for an ecological view of early help and support

Despite being ostensibly composed of a larger proportion of services with ‘gold-standard’ evidence of effectiveness, the system as a whole seems to be achieving less in regard to reducing rates of Children in Need than it was before 2015. This could be a consequence of underfunding in general, if any service stretched thin enough becomes unable to address more universal needs of the population and must consequently fire-fight more complex problems (Devaney, 2019; Hood et al., 2020a). However, these findings could also indicate some erroneous assumptions in child welfare policy.

The assumption that multiple high-quality interventions can be reliably scaled into an effective service ignores the complex ecology of child welfare services and children’s lives (Bronfenbrenner, 1979). While much attention has been directed towards answering what makes an effective intervention, comparatively little has addressed what makes an effective system; nor have the methods that are needed been developed in the ways they have in clinical studies. As a result, it becomes easy to make the assumption that preventative services as a whole are equal to the effectiveness of their individual programmes. However, the diversity of services may create compounding benefits for addressing family needs – policymakers may benefit from a willingness to tolerate services with diverse forms of evidence.

More ‘ordinary help’ provided over long periods of time may help families address underlying problems such as poverty and enable more productive engagement with interventions that address acute or complex needs. Ordinary help provision may also reduce the impact of failure demand on costly early interventions by providing practical and community resources that can prevent problems from escalating to the point they require referral to an acute or crisis service. Creating better harmony between intervening early in a child’s life and intervening early in the life of a problem, regardless of the age of the child or family circumstances, may create far more effective preventative systems on the whole than an intentional or unintentional focus on one or the other. Indeed, this might have happened if the Allen Report’s recommendations that evaluation quality should only guide new funding, and not be the basis for dismantling existing forms of provision, had been possible to follow. National policies, including the decimation of the local authority central grant, made this impossible, and ‘quality of evidence’ has become a bigger requirement for justifying the continued existence of many services. Holistic assessments of different forms of evidence, and a ‘systems-eye view’ through the use of methods like ALT-SR, are recommended to create effective ecosystems for addressing need without putting services that provide ordinary help at risk.

Limitations

We remain unable to reliably investigate more nuanced categories of expenditure to explore which types of spending may be most effective for reducing CIN rates. While this article focuses on cross-referencing these findings with the policy focus on effectiveness and early intervention of the past decade, child welfare and policy change over this period has been complex and multi-faceted, and explorations of alternative or additional explanations should be encouraged. Further, this study does not examine how preventative spending might have affected other outcomes over time. These services might have become more effective at reducing rates of other state interventions like child protection plans or child removal into state care, which are salient concerns for local authorities, though this would still imply a failure to meet duties to children under the Children Act 1989.

We are also unable to disaggregate CIN into more specific categories before 2012, such as into Children in Need because of a risk of neglect compared to Children in Need because of disability. This may result in underestimating the effect of spending on reducing maltreatment or neglect related risk, as children with disabilities remain ‘in Need’ under section 17 until adulthood. A profoundly different outcome measure is needed to assess the efficacy of preventative services for improving the lives of disabled children. Lastly, the study only observed one-year time lags, but it is possible that there are additional or cumulative effects over shorter or longer spans of time.

Conclusions

Early help and family support expenditure funds many services that may not conform well to randomised controlled trials and, as such, this puts potentially effective services at risk of retrenchment (Stewart-Brown et al., 2011; Stewart-Brown, 2012; White et al., 2014). Wider trends and individual case traits in administrative data mean that estimating system-level effects accurately can be difficult, and this is particularly acute in the case of assessing the impact of preventative spending on Children in Need rates. ALT-SR models are able to separate within-unit effects from between-unit effects beyond existing alternative models to address this problem. Doing so shows that investment in preventative support services was associated with significant and contextually large decreases in CIN rates before 2014/15: between 11.5 and 14.5 additional Children in Need per 10,000 per year within local authorities are attributable to adjacent-year effects of spending reductions under austerity between 2010 and 2015.

According to the UK Census, there were 11.3 million people aged under 18 living in England in 2011, suggesting an additional 13,000 to 16,500 children and young people each year between 2010 and 2015 were put or kept at risk of developmental or health impairments as a result of local authority funding cuts to early help and family support services, after adjusting for differential and changing thresholds over time. As many of these children will have recurrent ‘in Need’ episodes in future years but not all of these will be the result of further preventative services expenditure cuts, it is reasonable to expect there is a significant cumulative impact of preventative spending cuts on total CIN rates in later years, though this is impossible to accurately estimate. For example, if every child remained in need for the five year duration, they would account for between 1-in-8 (13.3%) to 1-in-6 (16.9%) of all 390,000 CIN in England in 2015. Investment in early help and family support can reduce rates of Children in Need if policies are able to create effective systems.

Despite the ambitions of the Allen Report (Allen, 2011) and the establishment of the ‘What Works Network’ to drive effectiveness in UK early intervention, it appears to have declined. These findings highlight the importance of routinely assessing local services as more than the sum of their parts and developing robust methods that enable such analyses. Going forward, the task of designing children’s services may be better served by identifying effective ecosystems of support and the way that their internal components work together. Intentionally or unintentionally designing systems by scaling-up interventions with ‘gold-standard’ evidence without consideration of wider contexts may be a poor basis for policy, as others forewarned (Stewart-Brown, 2011; White et al., 2014).

There are applications for ALT-SR models in social policy research beyond the example of child welfare services. Many areas of research use data at local government department, state, region, county, or country level that can be similarly confounded by larger individual differences and global trends that should be adjusted for to reliably estimate within-unit dynamics. ALT-SR’s flexibility within a structural equation modelling framework means that related between- and within-unit research questions can be explored simultaneously to test multi-layered hypotheses.