Shared Demographic Characteristics Do Not Reliably Facilitate Persuasion in Interpersonal Conversations: Evidence from Eight Experiments

Abstract

Many efforts to persuade others politically employ interpersonal conversations. A recurring question is whether the participants in such conversations are more readily persuaded by others who share their demographic characteristics. Echoing concerns that individuals have difficulties communicating across differences, research finds that individuals perceive demographically similar people as more trustworthy, suggesting shared demographics could facilitate persuasion. In a survey of practitioners and scholars, we find many share these expectations. However, dual-process theories suggest that messenger attributes are typically peripheral cues that should not influence persuasion when individuals are effortfully thinking, such as during interpersonal conversations. Supporting this view, we analyze data from eight experiments on interpersonal conversations across four topics (total N = 6, 139) and find that shared demographics (age, gender, or race) do not meaningfully increase their effects. These results are encouraging for the scalability of conversation interventions, and suggest voters can persuade each other across differences.

Across many contexts, politics features robust interpersonal advocacy between citizens (for example, Broockman and Kalla 2016; Foos and De Rooij 2017). These include door-to-door persuasion efforts by political campaigns (for example, Bailey, Hopkins, and Rogers 2016; Foos and John 2018), canvassing campaigns to reduce intergroup politically-related prejudices (for example, Broockman and Kalla 2016), in-person workshops, or one-on-one conversations between outpartisans, intended to reduce partisan animosity (for example, Baron et al. 2021; Santoro and Broockman 2022).

As many countries grow more diverse, a recurring question about these efforts is whether they are more effective when conversation participants share demographic characteristics such as race, age, or gender. For example, would an elderly white male be more persuaded about an issue by a canvasser with these same characteristics than by a young black female canvasser? This question has arisen among practitioners, many of whom assume such demographic concordance is valuable (Demlabs 2022); in studies of political campaigns, where evidence of backlash raises the possibility that discordant canvasser identities could be a negative cue for voters (for example, Bailey, Hopkins, and Rogers 2016; Enos and Hersh 2015), and in related literature on voter turnout efforts, which has wondered whether ‘canvassers who “match” the ethnic profile of [voters] tend to have more success’ (Green and Gerber 2019, 45–46).

However, despite this speculation in the existing literature, it is far from obvious whether shared demographic characteristics reliably facilitate persuasion. On the one hand, theories of source credibility suggest individuals perceive similar others as more trustworthy. On the other hand, dual-process theories of persuasion suggest that messenger attributes such as shared demographics are typically peripheral cues that should not facilitate persuasion when individuals are engaged in effortful thought such as interpersonal conversations, unless these cues are relevant. These and other theories suggest that – at least when they are speaking with each other – voters may be better able to look beyond demographic differences than some expect. Furthermore, using an original survey we find that practitioners and scholars hold heterogeneous expectations about whether demographic matches would facilitate attitude change, although most expect they would.

We examine whether shared demographic characteristics facilitate persuasion in interpersonal conversations using data from eight experiments across four distinct domains: transphobia, abortion, immigration, and partisan animosity. These studies involved a mix of in-person door-to-door interventions implemented by trained canvassers and face-to-face online video calls between untrained laypeople. In all studies, participants were randomly assigned to treatment or control groups; further, conversation partners were either fully randomly assigned or as-if randomly assigned to each other, facilitating causal inference on whether shared demographics moderate the persuasive impact of the conversations. In each study, we match the data on the demographics of study participants with records of their conversation partners' demographics and estimate effects when there are and are not demographic matches between them (for example, same gender). Finally, we combine these study-specific estimates in a meta-analysis.

Our meta-analysis found consistent null effects of demographic concordance on attitude change: the interpersonal interactions we studied did not have meaningfully different effects, regardless of whether the participants were or were not of the same race, gender, age, or any combination of these. In closing, we discuss the theoretical and practical implications of these results.

Theoretical Perspectives

Many generations of social scientists have examined the role of the source in conveying a message's credibility (see for example, Hovland and Weiss 1951; Zaller 1992), and broadly found that high-credibility messengers are more persuasive than low-credibility messengers (see Pornpitakpan 2004; and for where the effects of source credibility are strongest, see Tormala and Clarkson 2007; Kumkale, Albarracín, and Seignourel 2010). Recent literature has suggested that demographic characteristics such as race or gender can increase a source's credibility by bolstering the message receiver's perception of a persuader's expertise or trustworthiness (Pornpitakpan 2004). In particular, receivers may find in-group members more credible because of signals of shared experience or trust, or cognitive heuristics favouring the in-group (Tajfel 1970). Related literature on gender and race of interviewer effects (for example Kane and Macaulay 1993; White and Laird 2020) also suggest, more generally, that individuals respond differently based on interviewers' demographic characteristics. Moreover, some argue that these effects are more pronounced when the characteristic on which there is demographic concordance is made salient by the topic of conversation (for example, Davis 1997).

With that said, dual-process theories of persuasion suggest that messenger source cues, such as shared demographics, may not facilitate persuasion when individuals are engaged in effortful thought. In particular, the elaboration likelihood model of persuasion and the related heuristic-systematic model of persuasion both find that whether individuals rely on such ‘peripheral’ (also called ‘heuristic’) cues when processing a persuasive message depends on whether they are effortfully thinking about the message (Petty and Cacioppo 1984). When individuals are effortfully processing, they rarely rely on source cues, but instead tend to scrutinize the quality of arguments and evidence (Petty and Cacioppo 1984).

Previous research suggests these dual-process theories yield important insights into understanding the distinct dynamics of interpersonal conversations. Because interpersonal conversations require individuals to respond and react, they are likely to engage individuals in effortful thought (Kalla and Broockman 2020). In so far as interpersonal conversations do trigger more effortful processing, this research suggests that messenger cues such as shared demographics may be unlikely to facilitate persuasion in interpersonal conversations.¹ Consistent with this, not all laboratory- and survey-based studies find that shared demographics with a source facilitate persuasion (see, for example, Miller and Kurpius 2010).

These competing perspectives hold important theoretical and practical implications for understanding politics. In diverse democracies, can individuals persuade each other across lines of demographic difference, or would individuals and organizations be wise to carefully consider the demographics of their messengers, pairing demographically similar individuals? Such a strategy might limit the scalability of interpersonal conversation efforts, but increase their effectiveness to the extent demographic concordance facilitates persuasion.

Unfortunately, despite the theoretical and practical stakes of this debate, there is little data on whether shared demographic characteristics affect persuasion in the field. In the next section, we introduce the data we use to shed light on the role of shared demographics in facilitating persuasion in the field.

Data and methods

To conduct our analysis, we searched for studies, (a) that experimentally evaluated the effects of interpersonal conversations between two people, (b) that intended to politically persuade in some way, (c) where conversation participants' demographics were measured, (d) where participants were randomly or quasi-randomly assigned to each other, and (e) which took place either in-person or using video calls (so participants had clear signals of others' demographics). See Appendix B for examples of excluded studies and our study search procedure.

This criterion yielded the eight studies listed in Table A1, which span four issues and two modes. First, Broockman and Kalla (2016) and Kalla and Broockman (2020, Study 2) study the impacts of door-to-door canvassing interventions intended to increase support for transgender-inclusive non-discrimination laws and reduce anti-transgender prejudice. Next, Kalla and Broockman (2020, Study 1),² Kalla and Broockman (2023, Study 1), and Kalla and Broockman (2023, Study 2) study the impacts of canvassing interventions intended to increase support for various policies related to undocumented immigrants and to reduce prejudice towards undocumented immigrants. Next, Kalla, Levine, and Broockman (2022) study door-to-door canvassing related to abortion.

Finally, we also analyze Santoro and Broockman (2022, Studies 1 and 2), who studied video calls between Democrats and Republicans, intended to reduce partisan animosity.³ Unlike the other studies we analyze, these studies do not involve explicit attempts at persuasion from a trained canvasser; rather, all participants were untrained laypeople and the study investigates whether the interactions reduced partisan animosity. We analyze this study separately because it is conceptually distinct from the others, but include it to probe the generalizability of our findings to other forms of interpersonal interactions.

Together, the eligible conditions in the eligible studies include N = 6, 139 total observations. For studies with multiple effective canvassing treatment arms, we pooled treatments into a single treatment group to maximize statistical power. We also performed a power analysis to determine whether we would be well-powered to detect the cost-effective effect size for demographic concordance (0.10 standard deviations); we are well-powered to detect differences of this size for concordance on race, gender, age, and race-gender (see Appendix E).

All these studies feature both random assignment to treatment and control groups and random or as-if random assignment of conversation partners to each other. The latter is crucial for inference regarding the causal effects of partner demographics; otherwise, treatment effects might vary based on whether there is demographic concordance between conversation partners for reasons unrelated to the causal effect of partner demographics (for example, white canvassers might choose to canvass white neighbourhoods, and white neighbourhoods could be harder to persuade on some issues). To accomplish this, the first six studies all randomly assigned canvassers to voters. However, many of these studies took place across multiple sites and demographic concordance was more likely in some sites than others, so we used study-site fixed effects in some analyses so that all comparisons took place within sites. Santoro and Broockman (2022) randomized participants to treatment or control conditions but did not explicitly randomly pair participants with each other. Rather, researchers announced the time when participants should arrive for online video calls, and paired participants as they arrived, Because who participants were paired with was a function of second-by-second coincidences, we considered their assignment to conversation partners quasi-random.

One limitation of our analysis is that all of the eligible studies were conducted in the United States. However, as shown in Table A2, study participants are broadly diverse.

Because the canvassing experiments we analyze are conditioned on the study participants opening the door for the canvasser and identifying themselves to the canvasser, it is critical for causal inference that study participants who differ in their persuadability are not more or less likely to agree to do so with demographically concordant instead of discordant canvassers.⁴ For example, if more persuadable individuals are more likely to open the door and begin a conversation with individuals who share their demographics, those successfully canvassed by demographically concordant canvassers may have higher treatment effects, not because of the effects of demographic concordance itself, but because of the differing composition of those canvassed. We conduct two tests that help bolster our assumption that this does not occur. First, in Table A6 we show that there are no substantively or statistically significant differences between overall rates of door opening based on demographic concordance in the subset of studies where data on logs of both successful and unsuccessful contact are available (c.f. Michelson 2006). Second, Figure A2 shows that there is a good covariate balance between those canvassed by demographically concordant and disconcordant canvassers, consistent with voters opening the door for both groups of canvassers being exchangeable.

Estimation and Variables

To estimate whether demographic concordance facilitates persuasion, for each study-site combination we run regressions of the form: $Y_i = \alpha + \beta T_i + X_i + \epsilon _j$. The dependent variable Y _i is attitudes on the relevant issue for respondent i, using an index of all primary dependent variables, as used in the original studies. In studies with multiple rounds of post-treatment surveys, we use data from the first post-treatment survey. T _i is the randomized treatment indicator indicating whether participants received the conversation intervention or not. In Santoro and Broockman (2022), this is whether they were informed they were speaking with an outpartisan (which Santoro and Broockman (2022) find decreased partisan animosity); in the remaining studies, it is whether canvassers delivered a persuasive intervention. To increase precision, we control for the same vector of covariates pre-registered in the original studies, X _i.

We study whether demographic concordance between conversation participants, which we denote C, facilitates persuasion. To estimate this, we separately run the regressions in the above equation within each study in cases when there is demographic concordance between conversation participants (C = 1) and when there is no demographic concordance (C = 0) on the particular type of concordance being studied. For each study, this yields a conditional average treatment effect when there is demographic concordance and a separate conditional average treatment when there is no concordance.

To estimate the effects of C on persuasion, we run meta-regressions of the form $\beta _{k, C} = \theta _k + \gamma C_k + \epsilon$ where β _k,C are the estimates of the conditional average treatment effects for study k with at least ten observations, separately by whether or not there is demographic concordance (C). Our coefficient of interest is γ, the estimated effect of demographic concordance on the treatment effects of the interventions. θ _k study- or study-site level fixed effects, depending on the analysis, and $\epsilon$ is the error term.

We study three types of concordance: age (defined as the conversation partners being within fifteen years of each other), gender, and race. We also study all two- and three-way combinations of these demographics. In addition, for the studies in Santoro and Broockman (2022), we also study education concordance, defined as the conversation partners both being college-educated (a variable unavailable in the other studies). Table A3 lists which studies contain which variables.

Prior survey of academics and campaign practitioners

To contextualize our meta-analytic results, we fielded an online survey measuring expectations about the efficacy of canvassing when there is or is not demographic concordance. Our survey respondents were a small convenience sample of political practitioners with expertise in canvassing for political persuasion and academic researchers with expertise in attitude change, persuasion, or prejudice reduction. The survey of academics and campaign practitioners used the context of the canvassing studies as its motivating example. See Appendix C for more details.

Results

The results for the canvassing studies are presented in Fig. 1. The left-hand side plots the estimated treatment effect of there being demographic concordance versus not; that is, positive estimates mean that the effects of the interventions are estimated to be larger when there is demographic concordance. The right-hand side shows the predictions from the survey of academics and practitioners.

Figure 1. Effects of demographic concordance in canvassing studies.

Notes: In the left panel, each coefficient shows the estimated differences between the effects of intervention when conversation participants do and do not share the demographics listed on the left. Standard errors (thick lines) and 95 per cent confidence intervals (thin) surround point estimates. The right panel shows histograms from our survey of academics and practitioners.

For example, 80 per cent of academic and campaign practitioner respondents thought that persuasive effects would be larger when the race of the canvasser matched that of the voter. However, with the study's fixed effects, we estimate that shared race does not increase the effects of the interventions. The point estimate is 0.007 standard deviations with a standard error of 0.028. By contrast, the average treatment effect of the interventions themselves in the canvassing studies was approximately 0.10 standard deviations. The remainder of Fig. 1 shows that this pattern of null results holds across various forms of demographic concordance.

For key categories of concordance, for which our sample sizes are large – like race, gender, age, and the interaction of race and gender – we have sufficient statistical precision and power to detect the kinds of effects that are large enough that a canvassing organization would want to account for them when implementing a campaign (see Appendix E). For instance, the estimate (0.007) and standard error (0.028) on the effect of race/ethnicity concordance indicate that substantively meaningful differences between the effects of the interventions in cases when conversation partners do and do not share race and ethnicity are unlikely. Although we cannot rule out very small effect sizes for this type of concordance, we can statistically rule out large differences (as the top of the 95 per cent confidence interval is 0.06 standard deviations, compared to a baseline average effect of 0.10 standard deviations; see Table A7 for details on the statistical precision of estimates as well as a post hoc power analysis). The precision of our null estimates for gender is similar, with the top of the confidence intervals around 0.05 standard deviations. Our confidence intervals for race and gender concordance therefore both exclude increases of about 0.05–0.06 standard deviations, which is about half the size we calculate the effect would need to be in order to ensure that concordance would be cost-effective (see Appendix E). Although we cannot rule out prior beliefs that demographic concordance only slightly improves persuasion, many of these prior expectations would be inconsistent with cost-effective persuasion, and our point estimates are still lower than many of these prior expectations.

Because some other types of concordance are less common in the available data, we have less precise estimates and less statistical power for estimating the effects of concordance in terms of race-age, gender-age, and race-gender-age (for example, simultaneous matches on race, gender, and age occurs only 5 per cent of the time in the included studies). Our power analysis suggests that we are not well-powered to detect the cost-effective effect size for these types of concordance; see Appendix E. Therefore, we cannot rule out some large differences in the effectiveness of conversations based on the simultaneous match of multiple demographics. Fig. 1 shows the estimates, standard errors, and confidence intervals for these and our other estimates.

Limiting the results just to voters of colour – a group that may be expected to be particularly responsive to racial concordance in the US context – we continue to find null effects of the race of the canvasser matching those of these voters, although the standard error is large (effect estimate of −0.013 standard deviations; standard error of 0.087).

The results for the online video studies focusing on partisan effects from Santoro and Broockman (2022) are presented in Fig. 2. We similarly find consistent null effects from demographic concordance. The average effect of the intervention itself in these studies was approximately 0.33 standard deviations; the estimated effect of demographic concordance on the effectiveness of the interventions is substantively small and never statistically significant.

Figure 2. Effects of demographic concordance in partisan animosity online video call studies.

Notes: Each coefficient shows the estimated differences between the effects of intervention when conversation participants do and do not share the listed demographics. Standard errors (thick lines) and 95 per cent confidence intervals (thin) surround point estimates.

See Tables A4–A5 for the full meta-regression results. Figure A1 shows the results of the study; we do not see patterns whereby positive estimates from one issue area cancel out negative estimates in other issue areas to spuriously create a null on average. Appendix F shows that some null-but-suggestive results are unlikely to be concealing true effects.

Conclusion

Political practitioners and ordinary voters regularly try to persuade others politically, or reduce political animosity through interpersonal conversations. But in a diverse country, are such interactions most effective when people are conversing with others similar to them? Both a substantial body of prior theory, including work on source credibility and the literature on gender- and race-of-interviewer effects (see, for example, discussions in Kane and Macaulay 1993; White and Laird 2020) and the elicited expectations of many academics and practitioners, suggest that they should be. However, dual-process theories suggest that when individuals are thinking effortfully – as they often are in lengthy interpersonal conversations – they often look past these source cues to more effortfully consider a message's content. In line with this theory, our analysis of eight experiments on the effects of interpersonal persuasion interventions across a number of topics and multiple mediums of delivery finds no evidence that demographic concordance facilitates persuasion or attitude change.

Our study has several important limitations. First, although we consider a diverse group of eight studies that span multiple topics and delivery modes, existing studies are largely limited to interpersonal and social issues, and our conclusions could differ for other issues or candidate campaigns. Second, due to data limitations, we do not examine other potentially important dimensions of congruence such as political ideology or interest (for example, Enos and Hersh 2015), areas ripe for future research. Third, although existing studies already demonstrate that less effortful cognitive processing facilitates source effects (for example, Petty and Cacioppo 1984), without data from studies where demographic concordance did facilitate persuasion, we cannot be sure what the mechanism would be for any such effects (as we did not observe any). Fourth, the topic of discussion was often not relevant to the demographic matching variable; future work might vary whether or not the topic is relevant to the partner's identity (for example, comparing racial demographic concordance when talking about a race-related issue). Fifth, the data we were able to locate were limited to the US context, and future research should examine whether these effects generalize to other contexts. Finally, our results do not rule out that demographics and identities likely affected the conversations in many other ways, or, given the presence of statistical uncertainty, that there could be effects we were unable to detect. For example, understanding whether concordance on both gender and age or race and age changes the effectiveness of canvassing interactions is theoretically important, but because those types of fine-grained concordance are rare in our sample, we, unfortunately, lack the statistical power to rule out some potentially meaningful effects.

With that said, our study provides good news for civil society. Some worry that, in a diversifying country, people only listen to members of their own groups, distrusting others who do not look like them. We find more encouraging news – that the impact of conversations can transcend these differences.