Political communication in the real world: evidence from a natural experiment in Germany

Abstract

Various robust communication effects have been identified, but evidence is overwhelmingly based on artificial survey treatments with limited real-world insight. I conducted a natural experiment on the impact of the European–Turkey statement closing the Balkan route during the 2015/16 European refugee crisis in Germany. This design tests the lasting effect of the statement's framing on public sentiment. I identify treatment and control groups based on timing to demonstrate its effect on perceptions of the crisis, asylum attitudes, and policy preferences. Effects are largest immediately following the announcement but decline rapidly. This shows political communication can significantly change opinion within a limited time frame. This study enhances our understanding of real-world communication effects and offers a broadly applicable methodology.

1. Introduction

Political communication can provoke intense, immediate public reactions. Beyond extreme cases research has demonstrated the ability of political communication to frame issues (de Vreese, 2003), set the agenda (Shaw, 1979; Feezell, 2018), and induce processes of learning (Bode, 2016).

These studies frequently rely on survey experiments, which are typically carried out using designs with text-based treatments in which the effect is evaluated immediately after treatment exposure. Because these designs are unable to distinguish between short- and long-term effects, it is difficult to infer whether communication effects persist in the real world.

Scholars across different disciplines compensate for this limitation by presenting repeated stimuli, but their findings are inconclusive. While research in communication sciences argues that the effects of framing can be quite enduring (Tewksbury and Scheufele, 2009; Lecheler and De Vreese, 2011) other demonstrate that political messages are short lived (Chong and Druckman, 2010; Hill et al., 2013). Especially in social psychology research, the transient nature of communication effects has long been acknowledged (Cook and Flay, 1978; Baesler and Burgoon, 1994).

Furthermore, translating these results to real-world cases is difficult because experimental designs guarantee that a desired communication effect reaches their target audiences rather than being deflected by recipients who are exposed to a plethora of competing messages in the real world. Consequently, we might overestimate both the significance and persistence of communication effects (Kinder, 2007). In response several scholars argue that the literature on political communication should be expanded with studies based on real rather than constructed stimuli (Kinder, 2007; Lecheler and De Vreese, 2016).

The goal of this study is to address this gap and better understand how communication effects persist in the real world by employing a causal design. I use time variation in existing survey data to capture communication effects and the period during which they affect public sentiment.

Specifically, I examine public reactions to the European (EU)–Turkey statement given by the European Commission on March 18, 2016, during the 2015/16 European refugee crisis, widely recognized as the formal closure of the Balkan route. The statement reframed the issue and the focus of the debate around the European refugee crisis from a normative to a more pragmatic perspective focused on solutions to reduce inflows (Gürkan and Coman, 2021). Although the statement announced a policy to reduce the inflow of refugees to Europe, it took six months to produce the promised outcome. This scenario enables the isolation of the statement's framing effect from the impact of the actual policy outcome. Using survey data available through the German Internet Panel (Blom et al., 2017), I analyze the effect of the EU–Turkey statement on a day-by-day basis.

My results show that the EU–Turkey statement affected respondents’ perceptions and attitudes toward the handling of the crisis as well as policy preferences. Respondents were more likely to view refugee inflows as manageable and less supportive of security policies. The effects were largest for respondents surveyed one and two days after the statement, subsequently shrank and gradually disappeared.

Despite the high salience of the European refugee crisis and the potential solution it offers, the effect of the EU–Turkey statement we observe is only temporary. These findings strengthen the perspective that communication effects fade quickly in the real world. However, in the brief period when communication affects public sentiment, the consequences can be far-reaching. The findings presented here should be understood as a clear encouragement to better understand and further investigate the time frames within which communication affects public opinion using real-world cases.

2. Communication and the time dimension

Research has provided robust evidence for the profound effect political communication can have on public sentiment (Shaw, 1979; de Vreese, 2003; Bode, 2016; Feezell, 2018). Scholars have begun to investigate the persistence of communication effects (Chong and Druckman, 2010; Lecheler and De Vreese, 2011; Hill et al., 2013). However, given the significant implications that a possible decay of communication effects has for previous findings, we still need to better understand this dimension of communication effects, particularly how it manifests in real-world scenarios (Kinder, 2007; Lecheler and De Vreese, 2016).

Studies that discuss the decay of political communication more generally find quickly decaying effects. Hill et al. (2013) examine the time dimension of political communication using data from the effects of advertising in the 2000 presidential election and 2006 subnational elections in the United States and conclude that communication is unlikely to have long-term consequences unless people pay close attention to it. Chong and Druckman (2010) examine the endurance of competing messages and find that if messages are not received simultaneously, the effects of older messages decay quickly. However, in both cases, it is unclear whether the repetition of the stimuli presented by the authors replicates their participant's media consumption behavior.

Lecheler and De Vreese (2011) examine framing effects in a survey experiment over one day, one week, and two weeks. They demonstrate that framing effects persist and that the duration of framing effects is affected by a person's level of political knowledge.

Tracing the influence of media messages over time is not a new concept, and scholars, particularly scholars in social psychology, have argued that communication and farming effects are transient (Cook and Flay, 1978; Baesler and Burgoon, 1994). Here, two core concepts are used to understand the endurance of communication effects.

The memory-based perspective assumes that individuals develop their opinions cumulatively based on previous judgments and experiences which they store in memory (Hastie and Park, 1986). Constructed attitudes, on the other hand, are understood as on-the-spot opinions based on information available at the time. In this more recent perspective, opinions are potentially so context-dependent that there is no such thing as a real attitude. Instead, attitudes are the present state of a connected system of experiences (Wilson et al., 2000).

While these approaches have different perspectives on attitude formation, they both differentiate between more and less effortful processing to understand the longevity of communication effects. Scholars who take a memory-based perspective argue that individuals anticipate making a judgment of a specific item after getting a message. They weigh each piece of evidence as they receive it, changing an online tally up or down and store the resulting judgment in memory. If confronted with the necessity to make an unanticipated judgment, people generate opinions based on whatever information they can recollect (Hastie and Park, 1986).

In the perspective of constructed attitudes, persuasive communication, which invokes greater effortful processing, causes cognition to become more firmly established and thus remain accessible to affect attitudes for a longer period. New cognitions that have been exposed to less effortful processing may not become as firmly rooted and are therefore less likely to be remembered after a lengthy period. They may, however, still drive behavior during the brief period in which they are active.

Ultimately, both major perspectives on attitude formation suggest that citizens who engage in effortful processing while receiving a communicated message are more likely to be affected for a longer time. In contrast, those who process the message intuitively are more likely to be affected briefly (Hill et al., 2013).

Consequently, discerning which circumstances encourage people to engage in effortful information processing is critical for understanding the longevity of political communication. Kahneman's (2011) work on dual-processing theory describes the need for cognition, the tendency to think intuitively or effortfully about a presented problem or piece of information as an intrinsic human feature. However, individuals with an inherent tendency for effortful information processing are too few to explain broader behavioral patterns. Besides, the general need for cognition research has shown that issue salience can influence tendencies to engage in effortful thinking for a larger proportion of society. Citizens are more likely to engage in information processing if an issue is highly salient (Ciuk and Yost, 2016), especially in cases where compelling policy information is communicated (Boudreau and MacKenzie, 2014).

Salient issues occupy a central position in citizens’ belief systems and are tied to deeply held beliefs and identities. Consequently, citizen develop clear demands when it comes to salient issues (Esses et al., 1993; Sears, 1993; Zaller, 2012). Messages that successfully relate to these demands affect perceptions by altering voters’ perceptions of the circumstances surrounding their policy demands (Arnold, 1990; Chen and Luttig, 2019). This link between the message and the proposed outcome would be the strongest immediately following the announcement. As the novelty of the proposed policy wears off, the link between the announcement and the proposed outcomes weakens. Therefore, even when communicating on important issues, elite communication will only cause a brief shift in public opinion. In the long run, regardless of how much effort voters put into information processing, a policy change that meets citizens’ demands will be required for lasting opinion change.

3. Communication effects in a real-world setting

Despite their great utility, survey experimental designs have limitations when it comes to understanding how communication affects the public in the real world. Artificial treatments do not compete with other influences and guarantee that the respondent receives the desired message (Kinder, 2007). In response, researchers suggest that communication effects should be tested using real rather than artificial treatments (Lecheler and De Vreese, 2016).

An alternative approach to traditional experimental designs is to observe communication effects in a natural experiment retrospectively. A natural experiment allows us to exploit an issue that is salient in the real world and observe the impact of a related communication across time. Designs such as regression-discontinuity or difference-in-difference allow establishing causality retrospectively with existing data sources. Natural experiments have the advantage of eliminating the need to artificially reconstruct people's communication channels. Instead, they enable us to observe how communication affects people from the moment a statement or message is delivered.

A requirement for this design is a case in which we can identify the precise moment at which communication about a subject began. For this purpose, the scenario that I exploit is the 2015/16 European Refugee Crisis and EU–Turkey statement announcing the closure of the Balkan route. In 2015, an unprecedented wave of refugees arrived in Europe. Over 1.2 million first-time asylum applications were submitted in Europe, with Germany being one of the most popular destinations. Such large-scale human movement resulted from the “Arab Spring,” a succession of uprisings and civil conflicts in North Africa and the Middle East. In Germany, the crisis reached a climax in late 2015. At that time, it was a major challenge for the German government to deal with the high influx of asylum seekers (Singleton, 2016).

On March 18, 2016, the EU Heads of State and Turkey agreed to end irregular migration from Turkey to the EU. The so-called EU–Turkey statement vouched to replace disorganized migratory flows. It proposed safe and legal pathways to Europe for those entitled to international protection in accordance with EU and international law (European Council, 2016). Despite many critical voices, the EU–Turkey agreement was perceived at the time as the official closure of the Balkan route (e.g., Die Welt, 2016; Koelner Stadt-Anzeiger, 2016; Mitteldeutsche Zeitung, 2016; Tagesspiegel, 2016).

The narrative of the Balkan route closure was of crucial political importance for Germany as one of the main destinations for asylum seekers. The German government under the leadership of Angela Merkel had to grapple with the consequences of the refugee influx more than many other European countries. This was exemplified by the “Willkommenskultur” that marked the initial German response to the crisis, which later faced substantial pressure due to the massive scale of incoming migrants.

The EU–Turkey deal was intended to relieve this public pressure. It was presented as a means for managing the high number of refugees, allowing Germany to exert more control over the asylum process, effectively reducing the immediate strain on the country's resources and infrastructure. Although it initially reduced the influx of migrants that flowed through Greece, it did not end the crisis. In practice and politically, the agreement has not delivered on its promises. The number of asylum seekers returned to Turkey under the agreement has been inconsequential, leaving thousands stranded in Greece in overcrowded facilities, such as Lesvos's Moria camp. Turkey's commitment to the agreement has also faltered due to mounting domestic antipathy toward refugees and frustration over Europe's perceived lack of contribution (Gray Meral, 2023).

The case of the European refugee crisis and the EU–Turkey statement is ideal for a retrospective investigation of the time dimension of communication effects on public opinion. The announcement of the EU–Turkey deal reframes the issue, highlighting a potential solution to the crisis and responding to those citizens urging for greater migration control (Gürkan and Coman, 2021). Nonetheless, the proposed policy was substantially delayed in its implementation and ultimately did not serve as an effective tool to address the crisis. As such, any influence of the EU–Turkey statement on public sentiment likely arises from the communication about the policy rather than its implementation.

The European refugee crisis has been highly salient for a long time. Therefore, according to previous research, citizens can be expected to engage in effortful information processing when elites communicate on the issue and have a clearly articulated opinion (Boudreau and MacKenzie, 2014; Ciuk and Yost, 2016). As discussed in the previous section, this effortful information processing should, if it affects individuals at all, result in a lasting communication effect of the EU–Turkey statement. As a result, with the EU–Turkey statement, we can test whether political communication can have such a lasting effect even when the policy being communicated about demonstrably failed to achieve its objective.

The design, on the one hand, allows us to test the effect of communication with all the noise and distraction that individual occur in the real world and that artificial experimental designs do not account for, which may often lead them to overestimate communication effects. However, time confounders may also introduce significant biases as such in the following sections. Subsequently, I will discuss an elaborate strategy to test the effect of the EU–Turkey statement as well as control and test for effects of co-occurring events.

4. Empirical strategy

I apply the unexpected event during surveys design (UESD) formalized by Muñoz Mendoza et al. (2019) to capture the effect of the EU–Turkey statement on public sentiments. Similar to regression-discontinuity designs, UESD enables the identification of causal effects by exploiting the occurrence of an unexpected event during the fieldwork of a survey to estimate its causal effect on the relevant outcome by comparing the responses of the individuals interviewed before and after the event. Similar designs have been used, for example, to identify the effects of COVID-19-related lock-downs (Schraff, 2020; Bol et al., 2021) or terror attacks (Boydstun et al., 2018; Nussio et al., 2019).

Using UESD enables us to observe communication effects in a real-world context and, therefore, allows us to address the question set out in this study. However, the precise mechanism linking the occurrence of communication and shifts in public sentiment is situated within a black box. We cannot definitively identify, for instance, whether individuals explicitly read the EU–Turkey statement or whether the statement's content is disseminated through various media outlets or interpersonal communications. Therefore, this methodology effectively complements survey experimental designs, which have the reverse challenge. Experimental survey designs can disseminate the processes within the black box by artificially reconstructing the process from the moment a communication is issued to an individual's response using survey items, but they struggle to extrapolate how the effects identified within their artificial black box extend to real-world situations.

The event I exploit in this paper, the EU–Turkey statement was released by the European commission on the evening of March 18th (European Council, 2016). According to UESD, the treatment group comprises respondents exposed to the event, while the control group includes those not exposed to the event. The exposure to the EU–Turkey statement can be formally described as follows:

(1)$$X_{i} = \left\{\matrix{ \rm{1 \,\, if } \, {\it d_i} \geq {19 } \hfill \cr \text{0 if } {\it \, d_i} < {19};\; \hfill }\right.$$

where X _i describes the news exposure and equates to one if respondents were surveyed on March 19 or after. i describes respondents in the German Internet Panel (GIP) sample and d _i the day of survey completion. Identifying a causal effect based on a sample of respondents questioned before and after an event relies on multiple assumptions. The design used in this paper involves two main assumptions, as described by Muñoz Mendoza et al. (2019). The first is Excludability. Every discrepancy between respondents questioned before and after the incident is the sole product of the event. The interview time will only impact the dependent variable by an event at the defined cutoff point. Therefore, factors that render the pre- and post-event conditions different violate the assumption of Excludability.

The second key principle is Ignorability. The outcome for each respondent must be independent of the interview moment. For time to be a valid event instrument, the assignment to separate time values should be independent of the dependent variable. Respondents’ assignment to interview times should be as good as random.

To examine the deterioration of communication effects over time, I test my treatment variable across different sub-samples. These sub-samples are selected based on the date the respondent completed the survey. The first sub-sample includes respondents surveyed one day before or after the cutoff, while the 2nd–12th sub-samples include respondents surveyed up to 12 days before or after the cutoff. As a result, the control and treatment groups are always equally sized.

For each of these sub-samples, I will run, depending on the outcome variables scaling, a simple cross-sectional logistic or multinomial regression.

Inspired by King et al. (2000), this paper presents simulated predicted values to provide more reader-friendly measures and account for fundamental uncertainty. More precisely, I will calculate and present the differences in simulated predicted values between the treatment and control groups.

I use these models and predicted values to analyze the effect of the EU–Turkey statement on respondents’ perception of the crisis, their policy preferences, and voting intentions. I will discuss the operationalization of the different outcome variables more in the subsequent section.

5. Data

Survey data that provide a dense sample of respondents around the cutoff is needed to apply UESD and to measure the effects’ longevity. The GIP provides survey data for March 2016, enabling an analysis of the announcement's effect on public opinion across time. The sample was collected throughout March and comprised 3141 observations. Respondents were able to join the survey at any time during the month due to a comprehensive sampling strategy detailed in the following section. This 22nd wave of the GIP includes a question asking respondents to choose between pro-refugee policy and security policy with the question:

In your opinion, how should politicians deal with this possible dilemma? Either one can fulfill the moral obligation to help refugees from war zones to help. Or you can guarantee security in German society. ¹

The GIP also surveys opinions about Germany's ability to cope with refugee inflows using the following statement on a Likert scale of agreement. From here on, the question will be referred to as ability to cope:

Germany can cope with the challenges posed by the influx of refugees. ²

Additionally, the GIP includes a question surveying the general support for asylum on the same scale. Hereafter specified as asylum support:

Germany should maintain its policy of accepting refugees from war zones. ³

I create dummy variables from these questions to evaluate whether the treatment variable affects the respondents’ perception of the refugee influx. The dummies evaluate to one if a respondent agrees or strongly agrees to the statement and zero otherwise. For evaluating policy preferences, I create a three-level nominal variable that specifies if respondents favor a focus on refugee support, security policy, or both policies equally.

6. Threats to identification

Subsequently, I discuss various threats to the presented identification strategy, starting with threats to Excludability. The first potential origin of bias includes fluctuations in refugee inflows. They could be expected to change as the reduction of refugee inflow is the proposed aim of the announced agreement with Turkey.

Figure 1(a) shows that the number of refugees in March is only marginally lower than in February and on a par with numbers in April. We can only find a significant decrease in asylum applications from October 2016 after a slight increase in August and September. While monthly data do not exclude the possibility of variations throughout March, these findings strongly suggest that fluctuations in refugee inflows do not bias the presented measure.

Figure 1. Exogenous trends: (a) refugee numbers 2016 and (b) issue salience (Google Trends).

Other events occurring right before the announcement represent a further potential source of bias. These events include the Bautzen⁴ and Claußnitz attacks⁵ or the riots during new year's celebrations in Cologne⁶. Figure 1(b) provides a rough estimate of issue salience based on Google Trends data (see for testings of Google Trends data as a salience measure Mellon, 2013, 2014). The figure shows that while other issues were highly salient in mid-to-end of February, the Total Interest Over Time ⁷ fell to a low level in March. I conclude that these events are unlikely to have influenced my outcome variables during the period studied.

Another challenge to Excludability is the endogenous timing of the speech. The German government and the EU were under severe pressure to announce convincing measures to handle the inflow of refugees. A potential deal had the intention to satisfy public opinion and discourage support for far-right parties. However, if we understand the negotiations between the EU and Turkey as a bargaining process (Krumm, 2015), either side could not solely control the conclusion of the negations. The Total interest over time presented in Figure 1 also shows that the attention toward the event had peaked before the start of the summit. At the time of the summit attention toward the event is relatively lower and citizens are unlikely to have anticipated the exact timing of the announcement. Additionally, I will present a placebo test in the robustness section to further support the compliance with the assumption of Excludability.

The next major threat to identification is Ignorability. Ignorability is usually challenged due to insufficiently rigorous sampling strategies.

The GIP is mostly conducted as an online survey. Still, while online surveys are often associated with insufficient sampling and self-selection bias, the GIP uses a three-stage probability sample to counteract these issues. Primary sampling units (PSUs) are sampled stratified by state, government district, and urban level; for each PSU, interviewers list households along a random route with a random starting point. Each household along the predefined route was listed to prevent errors and interviewer cheating. Different interviewers conducted the listing and interview at all sample points. The listing yielded the third-stage sampling frame. A fixed set of addresses per PSU was drawn at random start intervals to minimize clustering (Blom et al., 2015). Considering this elaborate methodology, I assume sampling to be as good as random and Ignorability assumption to be fulfilled.

7. Descriptive statistics

Before the analysis, I descriptively examine the treatment and control groups in terms of their differences. Table 1 shows the p-values resulting from a two-sided t-test evaluating the significant difference of means between treatment and control groups across different demographic features and different bandwidths used in the analysis. Equally, it presents the size of the bandwidth sub-samples and the treatment and control group's respective sizes. I test several demographic characteristics that could shape the respondent evaluation of the ability to cope and their policy preference. The table shows that treatment and control only differ marginally. People aged 65 and older differ significantly in bandwidth 12, respondents with a higher education entrance qualification differ significantly between treatment and control in bandwidth one and respondents with a university degree differ significantly in bandwidth seven. Given that these two cases are the only ones where respondents are significantly different out of 72 total test runs, I concluded that respondents surveyed before and after the announcement are highly comparable in terms of demographic features.

Table 1. Treatment control comparison

Now that I have established the treatment and control group's comparability, I will examine the dependent variables around the cutoff point. Figure 2 shows the change in means for the output variables in March 2016.

Figure 2. Daily average for outcome variables.

From March 19 onward, we can see a significant change in averages in all three outcome variables. The ability to cope and asylum support outcome variables, in particular, follow a similar trajectory throughout the month. Following the EU–Turkey statement, the proportion of respondents who believe the refugee influx is manageable increased. On March 18, 35 percent of respondents thought the refugee influx was manageable, but 51 percent did after the announcement on March 19. Following this initial increase, we see a downward trend, with the lowest proportion on March 24, when only 17 percent of respondents saw the refugee flows as manageable.

Right after the European Commission announced the EU–Turkey statement, there is an increase in respondents who consider refugee support and security policy equally important in terms of policy preferences. On March 18, only 42 percent of respondents thought both policies were equally important, whereas 65 percent did on March 19. Simultaneously, the proportion of respondents who thought security policy was more important has dropped dramatically. On March 18, 36 percent of respondents thought security policy was the most important, while only 13 percent thought so one day later.

Overall, the descriptive analysis of the dependent variables indicates that the announcement has only a short-term effect on attitudes, which fades after a few days.

8. Results

Subsequently, I discuss various simulated predicted values of the effect of the treatment on the ability to cope, asylum support, and policy preferences. Figure 3 shows differences between treatment and control across the different bandwidths for all three outcome variables.

Figure 3. Difference in predicted probability. Bars show 95 percent confidence intervals. Regression tables can be found in the online Appendix.

The results for the outcome variables ability to cope and asylum support are derived from two sets of linear regressions conducted across different bandwidths. Only for the narrower bandwidths do respondents in the treatment and control groups have significantly different attitudes toward the corresponding outcome variable.

For the ability to cope the difference in predicted values is largest in the one-day bandwidth where people on average evaluate Germany's ability to handle the inflow of refugees 0.48 scale points higher. The value difference falls to 0.22 scale points in the five-day bandwidth until it is no longer significant from the sixth day on. For asylum support the results show that in the one-day bandwidth, respondents are on average 0.45 scale points more supportive of asylum policy. which falls to 0.2 scale points in the six-day bandwidth until there are no longer any significant results from the seventh day.

Consequently, the EU–Turkey statement affected the perception of the refugee influx to Germany, and the results support the idea of communication effect's short-lived nature.

I use multinomial models regressing the respondent's policy preference on the treatment variable to estimate the effect of the EU–Turkey announcement on policy preferences. The results provide evidence that the Balkan route closure's announcement affects policy preferences. Respondents surveyed one day after the information became public are on average 19 percent more likely to think support for refugees and security are equally important and 23 percent less likely to respond that security should be a priority. For wider bandwidths, the difference in predicted probability declines until it is no longer significant. Eventually, the treatment and control groups’ probability to think that both policies are equally important is no longer significantly different from the four-day bandwidth. Likewise, the treatment-control difference for security policy preferences is no longer significant from the eight-day bandwidth.

The announcement of the Balkan route's closure does not affect the likelihood of respondents favoring refugee policy. In combination with the negative effect on preferences for security policy, the Balkan route closure's announcement may mitigate the fears related to a higher level of migration rather than mobilizing support for refugees. Again, both previously defined expectations are met. The announcement of the EU–Turkey statement affected policy preferences but only temporarily. These results provide further evidence for the importance of time when trying to understand the effect of communication on public opinion.

9. Robustness checks

Figure 4 depicts a placebo test designed to address potential sources of bias within the control group. I shift the treatment cutoff date from the 12th to the 17th, using a one-, two-, and three-day bandwidth.

Figure 4. Placebo tests with difference in simulated predicted differences. Bars show 95 percent confidence intervals. Regression tables can be found in the online Appendix.

When the cutoff is set to the 13th, the placebo treatments show significant results. On that date, state elections were held in the German states of Rhineland-Palatinate, Saxony-Anhalt, and Baden-Württemberg. The Regional Elections pane in Figure 5 replicates Figure 3 while omitting respondents surveyed on the 13th and 14th to ensure that respondents polled briefly after these elections do not skew the presented results. The figure only includes the results from the five-day bandwidth as only these bandwidths include those respondents sampled directly after the state elections. The resulting predicted values differ only marginally from the original results, implying that the state elections on March 13th did not bias our results.

Figure 5. Robustness tests. Bars show 95 percent confidence intervals. Regression tables can be found in the online Appendix.

In addition, in the Asymmetrical Design section of Figure 5, I present an alternative design for bandwidth selection to the approach presented in Figure 3 to further demonstrate that the presented treatment effect is not biased by state-level elections. In this alternative approach, the control group is limited to a two-day bandwidth before the treatment, avoiding the inclusion of respondents polled in close proximity to state-level elections. The bandwidth for the treatment group has also been reduced to two days to ensure that treatment and control remain comparable. Subsequently, the bandwidth for the treatment group shifts from the 19th/20th to the 29th/30th while maintaining its two-day width. The results of this alternative approach are in line with the previous results and show significant effects in the first bandwidth for the ability to cope, Asylum Support, and policy preferences. They also demonstrate the same temporary nature of the effect, as the treatment no longer has a significant effect after the third bandwidth.

Underlying time trends are another potential challenge for UESD. The Time section of Figure 5 reproduces the results shown in Figure 3 while adding a time control variable. The findings indicate that the treatment and control groups are still significantly different. Furthermore, we can see that the effects are still diminishing over time, and the original argument remains valid.

In addition to the key outcome variables, I apply my design to an external outcome variable to see if the treatment effect is subject to random variation. The alternative outcome assesses attitudes toward imposing an environmental toll in German city centers, a policy preference that I do not believe the treatment will affect. The results are shown in Figure 6, and the treatment variable has no effect on the alternate outcome variable. These findings suggest that the previously discovered treatment effects are not subject to random variation.

Figure 6. Alternative outcome: support for an environmental toll. Bars show 95 percent confidence intervals. Regression tables can be found in the online Appendix.

I also run the model with demographic control to eliminate the possibility that the results are biased by respondent features. Dummies for respondents who live in East Germany, have higher education entrance qualification (Abitur), have a university degree, are unemployed, are over 65 years old, and respondent's gender are included.

In summary, the robustness checks presented show that the time dummy gauges exposure to the EU–Turkey statement. The potential bias I identified on March 13th as a result of state elections held in several German states have no significant impact on the results. The identified treatment effect is neither the sole expression of an underlying time trend nor the results of random variation.

10. Conclusion

This article investigates the persistence of communication effects in the real world by using a natural experiment approach to analyze the announcement of the Balkan route closure. Experiment-based studies have provided strong evidence on the significance of communication effects on public sentiment (Shaw, 1979; de Vreese, 2003; Bode, 2016; Feezell, 2018) and shed some light on how these effects play out over time. However, we heretofore knew very little about whether communication effects persist in real-world scenarios.

By utilizing time variation in survey sampling, the study at hand was able to provide an alternative to traditional survey experimental designs. A natural experiment allowed to analyze changes in public opinion in response to the EU–Turkey statement announcing the closure of the Balkan route. The announcement affected how people perceived the issue, their policy preferences, and their voting intentions. All three effects peaked right after the announcement and faded quickly over the following week.

Social psychology research suggests that the longevity of communication effects is driven by the level of cognitive engagement with the communicated message (Hill et al., 2013). Given the importance of the European refugee crisis, previous literature suggests that citizens are likely to engage in effortful information processing in the case of the EU–Turkey statement. Therefore, my findings suggest that even when people process a communicated frame very consciously, communication effects only cause temporary disruptions in public opinion. I expect this limitation of communication effects originates from the real political demands that need to be satisfied to achieve attitude change.

I concur with previous suggestions. Scholars should avoid presenting composite or average findings on communication effects that fail to discuss the degree of deterioration of the effect they present (Hill et al., 2013). Natural and survey experiments can complement one another in the study of political communication. Although approaches like UESD allow for real-world and temporal observation of communication effects, they do not disseminate the mechanism that links communication to changes in public mood. These mechanisms can be elucidated by survey experiments which have difficulty generalizing their findings to real-world situations. Ultimately, the research presented here can be viewed as the first in a series of potential studies that can aid in our understanding of the real-world persistence of communication effects.