1. Introduction
Election campaigns are an integral part of democratic representation. During campaigns, candidates present their policy priorities to voters. Voters select political candidates based on their issue positions, promises, and past performance. Candidates express their positions in words, and these positions can shape laws and policy outcomes (Beerbohm, Reference Beerbohm2016). Many classic theories of representation assume that candidates seek to follow their policy priorities once elected (Riker, Reference Riker1996; Mansbridge, Reference Mansbridge2003; Sulkin, Reference Sulkin2009). While these theoretical models are normatively appealing, it remains unclear whether they accurately reflect reality. We know little about the congruence between individual candidates' policy priorities during election campaigns and subsequent legislative behavior. This is surprising since the “standard” point of view in many normative theories of political representation is “individualistic”, focusing on the voter-candidate (or principal-agent) rather than voter–party relationship (e.g., Manin, Reference Manin1997; Rehfeld, Reference Rehfeld2006; Urbinati and Warren, Reference Urbinati and Warren2008; Mansbridge, Reference Mansbridge2009; Wolkenstein and Wratil, Reference Wolkenstein and Wratil2021). Once elected, do politicians prioritize the policies they emphasized during campaigns? Or are the statements made during elections in media appearances, candidate manifestos, leaflets, town hall meetings, and televised debates merely cheap talk?
Existing research on congruence between campaign communication and political actions mostly focuses on political parties, rather than individual candidates. Party positions and issue salience are communicated in party manifestos (e.g., Budge et al., Reference Budge, Klingemann, Volkens, Bara and Tanenbaum2001), speeches (e.g., Proksch and Slapin, Reference Proksch and Slapin2015), press releases (e.g., Gessler and Hunger, Reference Gessler and Hunger2022), and social media (e.g., Sältzer, Reference Sältzer2020). Comparing pre-election policy priorities with post-election policy outputs or outcomes sheds light on parties' ability to fulfill pledges. Numerous studies confirm that parties tend to fulfill a substantial portion of their election promises (e.g., Louwerse, Reference Louwerse2012; Zubek and Klüver, Reference Zubek and Klüver2015; Thomson et al., Reference Thomson, Royed, Naurin, Artés, Costello, Ennser-Jedenastik, Ferguson, Kostadinova, Moury, Pétry and Praprotnik2017). However, in elections, particularly elections using candidate-centered electoral systems, candidates propose their own election promises, and candidate characteristics often influence vote choice (Carey and Shugart, Reference Carey and Shugart1995; André et al., Reference André, Depauw and Martin2015). Yet, evidence regarding candidates' ability and willingness to work on priorities emphasized during their election campaigns remains scarce.
The predominant focus on parties rather than candidates is primarily due to the lack of comprehensive data at the candidate level. In most countries, parties commonly issue manifestos, while individual candidates typically do not. Consequently, this precludes the use of manifestos for analyzing the specific priorities expressed by individual candidates during election campaigns. Some candidates distribute leaflets (Trumm et al., Reference Trumm, Milazzo and Duggan2023). Many candidates, though not all, are active on social media (Castanho Silva and Proksch, Reference Castanho Silva and Proksch2022). Response rates to candidate surveys are often relatively low (Bowler et al., Reference Bowler, McElroy and Müller2020), and anonymized responses cannot be linked to candidates. Only comprehensive and comparable sources detailing politicians' policy priorities during campaigns enable us to juxtapose campaign rhetoric with subsequent actions.
In this paper, we assess the congruence between campaign communication and legislative activities through the analysis of official candidate manifestos and politicians' legislative leadership posts. We focus on the case of Japan, recognized as the longest-lived and most stable democracy in Asia (Lipscy, Reference Lipscy2023). Every Japanese candidate contesting in the district tier of national elections publishes short manifestos, which are distributed to all voters in their constituencies (Catalinac, Reference Catalinac2016a, Reference Catalinac2018; Crisp et al., Reference Crisp, Schneider, Catalinac and Muraoka2021). These candidate manifestos present a unique opportunity to study the congruence between pre-electoral policy priorities and legislative posts. Every Japanese household receives a printed copy of these documents via post, with many voters taking their content into consideration when casting their ballots. According to Japanese election studies, between 31 percent (2005) and 39 percent (2014) of all respondents noticed the candidate manifestos during the election campaign (Table A1). Securing legislative leadership posts in policy areas emphasized during the campaign would suggest that candidates' pre-electoral priorities are meaningful indicators of their legislative intentions.
We assemble and merge two datasets to test our theoretical expectations. Based on state-of-the-art transformer-based machine learning methods (Devlin et al., Reference Devlin, Chang, Lee and Toutanova2019; Wankmüller, Reference Wankmüller2022; Müller and Proksch, Reference Müller and Proksch2023), we identify policy emphasis in 48,877 statements from 1270 candidate manifestos released during five lower house elections in Japan between 2003 and 2014. Afterwards, we combine our new measures of issue emphasis during campaigns with a novel dataset on the legislative posts of Members of Parliament (MPs) from 2003 to 2017.
Our results demonstrate that a higher issue emphasis on a particular policy area in manifestos leads to a statistically and substantively significant increase in the probability that an MP obtains a post in that area. We also detect differences across issue areas. The congruence is higher for distributive issue areas, such as Economy, Trade, and Industry; Land, Infrastructure, Transport and Tourism; and Agriculture, Forestry, and Fisheries, than for public goods and high policy areas like Internal Affairs and Financial Affairs (Pekkanen et al., Reference Pekkanen, Nyblade and Krauss2006). Furthermore, a measure of portfolio importance derived from expert surveys reveals weaker relationships for more important policy areas. These results imply that MPs, who focus on constituency-targeted particularistic policies, may strive even harder to secure legislative posts in these areas, possibly to enhance their popularity and re-election chances.
The findings carry important implications for theories of representation, campaign communication, and the congruence between elections and legislative actions. Campaigns act as meaningful indicators of subsequent legislative actions, since voters can anticipate the policy areas their representatives will be active in if they elect them. These findings affirm the validity of the mandate model of democracy and contest the cynical, yet commonly held, belief that politicians disregard their promises (ISSP Research Group, 2018). Moreover, the study contributes to debates about electoral reforms. Many countries have “personalized” their electoral systems, allowing voters to express their preferences not only for parties, but also for individual candidates (Renwick and Pilet, Reference Renwick and Pilet2016). Our findings confirm existing research suggesting that the personalization of electoral systems tends to incentivize candidates to focus on the preferences of their voters (e.g., Zittel et al., Reference Zittel, Nyhuis and Baumann2019; Schürmann and Stier, Reference Schürmann and Stier2023). Overall, the results emphasize that campaigns function as a crucial democratic mechanism, bridging policy matters between the electoral sphere and the legislative process.
2. Theory and hypotheses
Campaign communication plays a crucial role in the chain of accountability between voters and politicians. Campaigns “open up channels of communication that allow us to hold representatives answerable for their attempts to solicit our trust. If reliable, these channels enable us to elicit commitments that manage the power imbalances built in representative democracy” (Beerbohm, Reference Beerbohm2016: 382). Vote-seeking MPs face incentives to put their words into action, as the performance of individual MPs can sway voter choice, particularly in candidate-centered electoral systems (Cain et al., Reference Cain, Ferejohn and Fiorina1987; Carey and Shugart, Reference Carey and Shugart1995). We posit that MPs develop policy priorities according to the policy interests of voters in the districts where they are elected. In other words, MPs' policy priorities may derive from their voters' interests. This is attributed to the following mechanism: (1) voters vote on the basis of their evaluation of MPs as well as their evaluation of parties (Cain et al., Reference Cain, Ferejohn and Fiorina1987; Carey and Shugart, Reference Carey and Shugart1995); (2) MPs seek to represent the interests of their voters to win elections (Shepsle, Reference Shepsle1978; Weingast and Marshall, Reference Weingast and Marshall1988); and (3) different MPs seek to represent interests in different policy areas because voters' interests vary by district due to demographic differences (Adler and Lapinski, Reference Adler and Lapinski1997; Raymond and Holt, Reference Raymond and Holt2019).
Political theorists, journalists, and many voters doubt that campaigns provide a truthful signal of policy priorities. The cross-national survey by the ISSP Research Group (2018) reveals that only a minority of citizens believe that politicians try to keep their promises. Media coverage of campaign promises tends to focus on broken promises rather than fulfilled ones (Müller, Reference Müller2020), and political theorists assert that campaign communication may not convey the truth (for a discussion, see Beerbohm, Reference Beerbohm2016). For example, Manin (Reference Manin1997: 180) states: “[e]ven assuming that voters choose to pay some attention to the candidates' promises, they know, or should know, that the credibility of those promises is an open question. It is not reasonable on their part to suppose that candidates will necessarily honor their commitments.” Measuring candidates' pledge fulfillment proves difficult, especially in parliamentary systems. Party unity in legislative voting is usually very high (Sieberer, Reference Sieberer2006), making the task of tracking individual legislators' influence on bills quite challenging. In the absence of reliable indicators of candidate-level pledge fulfillment, Sulkin (Reference Sulkin2009: 1094) investigates the linkage between the content of candidates' televised advertisements and bill introductions as well as co-sponsorship in the US House. Our study follows a similar logic. To be clear, while we cannot measure promise-keeping by candidates in the same way it is measured at the party level (Thomson et al., Reference Thomson, Royed, Naurin, Artés, Costello, Ennser-Jedenastik, Ferguson, Kostadinova, Moury, Pétry and Praprotnik2017; Brouard et al., Reference Brouard, Grossman, Guinaudeau, Persico and Froio2018), the involvement in law-making through leadership posts serves as a signal of legislative action.
One of the most efficient ways for individual MPs to work on their priorities during campaigns is to hold leadership posts. Policy-making authority in governments, legislatures, and parties is delegated to sub-organizations, such as cabinet ministries, legislative committees, and party policymaking organs. In many democracies, leadership posts represent one of the few institutionalized channels for influencing law-making in a specific area. The distributive theory of the U.S. Congress demonstrates that legislators seek to serve on committees relevant to their constituents' interests. By holding committee posts, legislators can represent their constituents' interests and improve their re-election chances (Shepsle and Weingast, Reference Shepsle and Weingast1987; Weingast and Marshall, Reference Weingast and Marshall1988). The distributive theory has been extended and confirmed in other countries with strong parties beyond the United States, including several European countries and Japan (Fujimura, Reference Fujimura2012; Gschwend and Zittel, Reference Gschwend and Zittel2018; Raymond and Holt, Reference Raymond and Holt2019; Mickler, Reference Mickler2022). By holding legislative leadership posts, MPs can participate in policymaking and influence policy in each area. Although the party leadership retains the formal power to allocate legislative posts to MPs, obtaining desired posts necessitates investing time and effort, such as attending committees regularly and demonstrating policy expertise to the leadership. Our first hypothesis therefore states that candidates have electoral incentives to take on legislative posts in policy areas that they mentioned more often during their campaign.
Hypothesis 1: MPs who focus extensively on a specific policy issue in their campaign are more likely to take on legislative posts in the same policy area once elected.
The second hypothesis addresses how differences in issue areas shape the level of congruence between campaigns and legislative posts. Some policies have particular effects on a small group of constituents (i.e., distributive policy areas), while others have broad effects on most constituents (i.e., programmatic policy areas). We argue that this difference in the scope and target of policy effects across areas leads to differences in the congruence across areas.
To examine differences across policy areas, it is essential first to classify them. Prior work separates policy areas into distributive and programmatic policy areas, depending on whether policies affect particular or broad constituents (e.g., Cox and McCubbins, Reference Cox and McCubbins2007; Primo and Snyder, Reference Primo and Snyder2010; Catalinac, Reference Catalinac2016a, Reference Catalinac2016b). Pekkanen et al. (Reference Pekkanen, Nyblade and Krauss2006) and Shugart et al. (Reference Shugart, Bergman, Struthers, Krauss and Pekkanen2021) propose a more fine-grained three-policy classification, further dividing programmatic policy areas into “public goods” and “high policy” areas. We also adopt the three-policy classification, as this provides a more nuanced understanding of programmatic policy areas.
According to Shugart et al. (Reference Shugart, Bergman, Struthers, Krauss and Pekkanen2021: 35), distributive policy areas involve benefits that “can be targeted to specific geographical entities.” MPs can gain support from constituents in exchange for delivering agricultural or business subsidies to particular sectors and public projects to particular geographic regions. Public goods and high policy areas, in contrast, encompass benefits that affect the entire country, and individual MPs have a limited role in delivering them. Public goods areas involve “diffuse but specific policy benefits for [...] larger groups of citizens and organized interest groups”, such as “healthcare, education, public utilities, and environmental protection” (Shugart et al., Reference Shugart, Bergman, Struthers, Krauss and Pekkanen2021: 34–35). High policy areas are “those in which the party leadership is deeply engaged”, such as “management of the economy, foreign and defense affairs, and the broader functioning of the legal and constitutional system” (Shugart et al., Reference Shugart, Bergman, Struthers, Krauss and Pekkanen2021: 34).Footnote 1 In short, following Pekkanen et al. (Reference Pekkanen, Nyblade and Krauss2006: 189) and Shugart et al. (Reference Shugart, Bergman, Struthers, Krauss and Pekkanen2021), we propose that distributive areas comprise construction, transportation, trade and industry, agriculture, local affairs, house budget, and posts and telecommunications; public goods areas comprise environment, science, labor, social affairs, and education; and high policy areas comprise finance, foreign affairs, legal affairs, defense, cabinet, and taxation.
We predict a higher degree of congruence between campaigns and leadership posts in distributive areas than in public goods and high policy areas. Individual MPs have greater autonomy and effectiveness in securing distributive benefits. By targeting particular regions or sectors, individual MPs can achieve distributive benefits with relatively limited budgets. Furthermore, the effects of targeted benefits on other constituents are minimal (Cox and McCubbins, Reference Cox and McCubbins2007: ch 8). For example, distributing subsidies and public works projects to some constituencies has little impact on other constituencies, unless they are enormous projects that could strain national finances. MPs can engage in vote trading, in which they gain support for their own interests in exchange for supporting the interests of other MPs (Weingast and Marshall, Reference Weingast and Marshall1988). Consequently, MPs can effectively deliver distributive benefits, such as subsidies or public projects, to their own constituencies by participating in the policymaking process in governments, parliaments, and parties.
Individual MPs play a limited role in achieving public goods and high policy objectives, because governments typically adopt a unified policy applicable to the entire country. For instance, individual MPs face difficulty when attempting to reform a national pension system alone, as the system affects all constituents and thus requires the involvement of most MPs. Similarly, their impact on improving relations with other countries is limited when acting alone, as these are national-level issues. In short, we expect MPs to be more likely to follow their campaign priorities in distributive policy areas as opposed to public goods and high policy areas.
Hypothesis 2: Congruence between issue emphasis during campaigns and subsequent legislative posts is higher for distributive policy areas than for public goods or high policy areas.
3. The case of Japan
Having derived our testable hypotheses, we turn to the description of our data: candidate manifestos and legislative activities in Japan. Here, we outline the importance of candidate manifestos and describe relevant legislative posts.
3.1 Campaign communication in Japanese candidate manifestos
In Japan, every candidate running in the district tier in national elections is required to publish a concise election manifesto. These official documents offer a unique opportunity for scholars to identify MPs' policy positions and their salience on issues during elections (Catalinac, Reference Catalinac2016a, Reference Catalinac2018). Since 1996, the Japanese lower house has had a combination of Single Member Districts (SMDs) and Proportional Representation (PR). Around 300 members are elected from SMDs, and around 180 candidates are elected from a PR list. Candidates can run for both SMD and PR tiers; those defeated in SMDs have a second chance to be elected from a PR list. Voters have two votes: one for a candidate in an SMD and another for a party in PR. Candidates from major parties run for both tiers, and most aim to win in an SMD (Ariga et al., Reference Ariga, Horiuchi, Mansilla and Umeda2016). SMD systems incentivize candidates to cultivate both personal and party votes (Carey and Shugart, Reference Carey and Shugart1995). Therefore, under the Japanese mixed system of SMDs and PR, candidates are incentivised to develop their personal reputation with voters while also cultivating their party's collective reputation (Horiuchi and Saito, Reference Horiuchi and Saito2003; Krauss and Pekkanen, Reference Krauss and Pekkanen2011; Goplerud and Smith, Reference Goplerud and Smith2023).
Japan's Public Offices Election Law determines that the Election Administration Commission in each prefecture issues an election bulletin containing individual candidates' names, careers, and political views every national election (Article 167). The Commission distributes the bulletins to all households at least two days before an election (Article 170). Candidate election manifestos are essential for candidates to communicate policy areas and provide a comparable source of candidates' campaign communication. In fact, several studies have used candidate manifestos to measure electoral incentives and policy interests in the case of Japan (Tsutsumi, Reference Tsutsumi1998, Reference Tsutsumi2002, Reference Tsutsumi2013; Shinada, Reference Shinada2001; Reference Shinada2011, Reference Shinada2018; Catalinac, Reference Catalinac2016a, Reference Catalinac2016b, Reference Catalinac2018; Muraoka, Reference Muraoka2018; Crisp et al., Reference Crisp, Schneider, Catalinac and Muraoka2021).
3.2 Appointment of legislative posts
The Liberal Democratic Party (LDP) has been in power in Japan continuously since 1955, except for brief periods between 1993 and 1994 and 2009 and 2012. Under the LDP government, as in many other countries, policy-making authority in the cabinet, the party, and the Diet is delegated to sub-organizations, such as cabinet ministries, party policy divisions, and Diet committees. MPs who hold leadership posts in these organizations can influence policymaking (Tatebayashi, Reference Tatebayashi2004; Pekkanen et al., Reference Pekkanen, Nyblade and Krauss2006; Fujimura, Reference Fujimura2015; Rehmert and Fujimura, Reference Rehmert and Fujimura2024).
The policy-making process in the LDP government starts with a bill drafted by the cabinet.Footnote 2 Cabinet-sponsored bills (cabinet bills) account for most enacted bills, although both the cabinet and Diet members can submit bills. The cabinet comprises 12 ministries which play a central role in drafting cabinet bills. Each ministry is headed by a minister (daijin), a senior vice-minister (fuku-daijin), and a vice-minister (daijin-seimukan). Second, the LDP screens cabinet bills before submitting them to the Diet. Fifteen policy divisions in the Policy Research Council (PCR) play a central role in screening bills and influencing their contents. The cabinet cannot submit bills to the Diet without the party's approval. Using this veto power, PRC divisions can force the cabinet to revise drafts of bills in their favor. Three leadership posts – a director (bukaicho), an acting-director (bukaicho-dairi) and a deputy director (fuku-bukaicho) – screen bills in each division. Third, in the Japanese Diet, which adopts a committee-centered system, committees play a substantial role in deliberating bills (Fujimura, Reference Fujimura2012). Twelve standing committees organized by policy area (corresponding to the twelve cabinet ministries) have the authority to pass, abandon, or reject bills in each policy area under their jurisdiction. With only a few exceptions, bills that are not voted on or rejected by a committee are not sent to the parent chamber. A chair (iincho) and directors (riji) set the agenda in a committee, including decisions about whether to open a committee meeting, as well as selecting the bills for voting and scheduling when these votes will occur.
Having outlined the relevance of holding legislative posts, we briefly describe the appointment of legislative posts in Japan (see also Nakakita, Reference Nakakita2018: 70–81). In the LDP, MPs are required to submit a “request form” indicating the posts that they wish to hold to the party secretary-general every time posts are reshuffled. These posts include cabinet senior vice-ministers and vice-ministers, Diet committee chairs and directors, and party PRC division directors, acting-directors, and deputy-directors. Based on the requests from MPs, the LDP party secretary-general, the chief cabinet secretary, and ministers determine the appointment of senior vice-ministers and vice-ministers; the LDP secretary-general determines the appointment of Diet committee chairs; the LDP Diet affairs committee chairperson determines the appointment of Diet committee directors; the LDP PRC chairperson determines the appointment of PRC division directors, acting-directors, and deputy-directors. Meanwhile, the influence of intraparty factions on the appointment of legislative posts has significantly waned. In cases where multiple MPs request the same post, government and party leaders prioritize MPs with policy expertise in relevant policy areas. Therefore, legislative posts are a suitable proxy to determine whether MPs seek to deliver on their campaign promises in each policy area, as they represent MPs' intentions to implement policies and their policy expertise in that area.
4. Data and measurement
We combine human coding and transformer-based machine learning classifiers to construct a novel dataset of issue emphasis in Japanese candidate manifestos. We collected all the manifestos of the largest governing party candidates who ran and won a seat in SMDs in lower house elections between 2003 and 2014 (LDP candidates in 2003, 2005, 2012, and 2014 and DPJ candidates in 2009). We then match the emphasis on policy issues with a novel dataset on MPs' legislative posts. Our period of analysis begins with the 2003 election because Diet committees and cabinet ministries were restructured in 2001. Below, we describe the classification procedure, validation, and measurements of legislative priorities.
4.1 Classifying issue importance in candidate manifestos
The classification of issue importance in candidate manifestos consists of a multi-stage process. Previous studies have analyzed candidate manifestos using topic models (Catalinac, Reference Catalinac2016a) and scaling methods (Catalinac, Reference Catalinac2018; Di Tella et al., Reference Di Tella, Kotti, Le Pennec and Pons2023). While topic models are helpful for exploring a text corpus and unidimensional scaling models can measure candidates' ideological positions, such methods do not suit the purposes of this research question. Since we know the categories of interest in advance (11 policy areas of committees and ministries), a supervised classification trained through human annotation of statements is more appropriate than an ex-post interpretation of unsupervised methods.Footnote 3
The data collection, classification, and aggregation works as follows (see also Figure A3). First, we collect all available manifestos of the largest government party in 2003, 2005, 2009, 2012, and 2014. The average manifesto includes between 30.6 (2005) and 54.8 (2014) statements (Table A6). For comparability, we limit the analysis to members who served for the entire legislative term, excluding MPs who died, resigned, or left the party during the legislative term. After applying this restriction, our analysis considers between 95.6 percent (2014) and 100 percent of MPs (2009, 2012) elected in an SMD or those not elected in their SMD but via a PR list. We construct a text corpus of these manifestos and segmented the documents to the level of statements (usually full sentences or items from a bulleted list). We randomly select 60 manifestos and manually segment these documents into statements. We then compare the automated separation based on punctuation characters and bullet points with human segmentation (Figure A4). The number of manifestos statements identified through the automated segmentation correlates very highly with the segmentation of the same set of documents by human coders (r = 0.96).
Having validated the segmentation of Japanese manifestos, we turn to the classification of policy areas. The Diet and the cabinet are each divided into 12 policy areas. Our categories in the supervised classification task mirror these policy areas. We also add a No/Other Policy Area category. SI Section G includes detailed coding instructions and examples for each policy area. Two trained native speakers conducted various reliability tests. Intercoder reliability reached satisfactory levels (Krippendorff's α = 0.84) in the second coding round of 300 statements. Having trained the coders and validated the intercoder reliability, each coder coded 1500 statements, resulting in a combined sample of 3000 annotated statements.
We use bert-base-Japanese, a BERT (Devlin et al., Reference Devlin, Chang, Lee and Toutanova2019) model that was initially pretrained on the entire Japanese Wikipedia corpus as of September 2019. We fine-tune the BERT classifier for our prediction task using a randomly sampled training set of 2000 statements, with an additional 500 statements for evaluation during the fine-tuning process. The remaining 485 statements are then used for out-of-sample predictions.Footnote 4 The F1 scores (the harmonic mean of precision and recall) for the BERT classification range from 0.53 for Committees on Cabinet to 0.89 for Agriculture, Forestry, and Fisheries. The average F1 score across the eleven categories is 0.73. The performance metrics in Table A3 demonstrate that our fine-tuned BERT model substantially outperforms conventional bag-of-words classifiers.
Moving beyond individual statements as our unit of analysis, we calculate the proportions of each policy area for a separate held-out test set and compare the proportions with human coding of the same set of statements. Figure 1 shows that the aggregated proportions are similar, and the correlation between both measures is 0.97 (for a comparable validation, see Soroka and Wlezien, Reference Soroka and Wlezien2022; Müller and Proksch, Reference Müller and Proksch2023). Even though we observe misclassification at the level of individual statements, these errors seem to cancel each other out when aggregating issue salience to the level of manifestos. The predictions of the machine learning model closely mirror human coding and provide accurate representations of policy priorities in candidate manifestos.
Figure 1. Comparing the correspondence between the frequencies of policy areas in the test set based on the BERT predictions and human coding of the same set of statements.
We assess both the face and content validity of the classification by implementing a technique known as keyness analysis (e.g., Severin et al., Reference Severin, Strinzel, Egger, Barros, Sokolov, Mouatt and Müller2023; Zollinger, Reference Zollinger2024). This identifies distinct words classified into a policy area, as compared to statements classified into different categories. The keyness analysis (Table A4) underscores that our fine-tuned model works for the Japanese context, since almost all the top words for each category are relevant to the respective policy area.
After validating the text segmentation and classification, we aggregate the corpus of 46,961 statements to the level of manifestos and calculate the proportion of manifestos falling into each of the eleven policy areas. While the size of the leaflet is standardized, candidates use various layouts and font sizes, which results in different amounts of statements. 58.1 percent of the hand-coded statements and 58.8 percent of the entire text corpus are classified into one of the eleven policy areas (Figures A5 and A6). The No/Other Policy Area statements primarily focus on aspects like a candidate's name, determination, personality, or profile, such as education and previous occupations or endorsements from other politicians. In SI Section D, we annotated all sections of three typical manifestos to highlight the prevalence of statements unrelated to policy priorities.
For each of the 1744 hand-coded statements related to policy, we identified the type of content. 752 (43.1 percent) of the statements were classified as a campaign pledge (Thomson et al., Reference Thomson, Royed, Naurin, Artés, Costello, Ennser-Jedenastik, Ferguson, Kostadinova, Moury, Pétry and Praprotnik2017; Müller, Reference Müller2020), 628 (36.0 percent) of the statements are policy content but not a promise, 229 (13.1 percent) of the statements are clarifications, and 98 (5.6 percent) of the statements describe former jobs or are personal statements. Only 37 statements, corresponding to 2.1 percent of the statements, were classified as credit claiming (Table A5). Candidates use the limited space of their manifestos to communicate priorities for the upcoming legislative cycle rather than focusing on their achievements in the past. This is an important finding in itself since manifestos released by political parties tend to devote considerable space to describing the status quo, claiming credit, and discussing other parties' failures (Müller, Reference Müller2022).
4.2 Measuring legislative priorities
To measure candidates' legislative priorities, we manually collected information on assignments to leadership posts in Diet committees, party policymaking divisions, and cabinet ministries in the legislative cycles of 2003–2005, 2005–2009, 2009–2012, 2012–2014, and 2014–2017. The Diet and the cabinet each have 12 committees and 12 ministries, respectively, whose jurisdiction corresponds unambiguously to the categories in our fine-tuned transformer model.Footnote 5
We focus on five types of posts. The first type of post is the Committee Post, which indicates whether an MP served as chair or director in a committee. The second type of post is the Party Policy Divisions Post, indicating whether an MP served as director, acting director, or deputy director in an LDP division. We do not focus on posts in the Democratic Party of Japan (DPJ), as the DPJ government, unlike the LDP, did not have a bill screening process within the party. The third post is the Cabinet Post, indicating whether an MP served as state minister or vice-minister in the cabinet. We also assigned Ministerial Posts to each policy area. Legislative Post (Combined) refers to MPs holding a post in a committee, a party policy division, and/or a cabinet ministry. Most of these posts are reshuffled annually. Table A2 provides details about the number of legislative posts in each policy area.
4.3 Dependent and independent variables
Our dependent variable is a binary indicator expressing whether an MP held a legislative post in a policy area. It is important to note that obtaining leadership posts is a competitive process: 31 percent of the MPs in our sample did not hold any leadership post for the eleven policy areas considered in the analysis during a legislative cycle (Figure A1). Across the five elections, 25 percent of MPs obtained several posts in the same policy area during one legislative cycle. Holding more than one post in one area was usually a consequence of internal promotions during annual reshuffles. Following Pekkanen et al. (Reference Pekkanen, Nyblade and Krauss2006), we opt for a binary indicator, i.e., holding at least one post in a given policy area, as the most appropriate indicator of legislative leadership. As we show in SI Section E.5, our results remain consistent when we predict the number of posts in one area (ranging from 0 to 3) using ordered logistic regression models.
The primary independent variable, Manifesto Salience, quantifies the focus on each policy area. It is calculated by dividing the number of statements specifically addressing a policy area by the overall number of statements that are relevant to any policy area. A simple example clarifies this approach: suppose a manifesto consists of 50 statements, and 40 statements are classified into one of the substantive policy areas. For instance, if ten out of these statements are about environmental policies, the variable would take the value 10/40, corresponding to 0.25 or 25 percent. Our results remain unchanged when using the count of statements about each policy area in a manifesto (SI Section E.6).
We test Hypothesis 2 with several indicators of policy importance. First, we include an interaction effect between Manifesto Salience and the type of policy area: Distributive Policy Areas, High Policy Areas, and Public Goods Areas (see Pekkanen et al., Reference Pekkanen, Nyblade and Krauss2006 and the discussion above). Second, we conduct individual models for each of the policy areas (resulting in eleven models per legislative leadership post). Third, we use Portfolio Importance as an alternative measure for assessing the perceived significance of policy areas. This metric reflects the level of importance experts attribute to each portfolio. For each lower house election, Junko Kato conducted an expert survey to evaluate the relevance of various portfolios (see, e.g., Kato and Laver, Reference Kato and Laver2003). Portfolio importance can range from 1 to 5, with higher values indicating higher relevance. We use the average score across all experts who completed each survey. The average portfolio importance ranges from 2.3 to 4.6, with a mean of 3.5 and a standard deviation of 0.73 (Figure A2). High policy areas rank highest in portfolio importance with an average of 4.2, followed by distributive policy areas at 3.2, and public goods areas at 2.9. We could match nine of the eleven policy areas to portfolios included in the expert surveys.
The selection of control variables follows prior work on legislative behavior and posts in the Japanese context (e.g., Pekkanen et al., Reference Pekkanen, Nyblade and Krauss2006; Fujimura, Reference Fujimura2015; Smith, Reference Smith2018). First, we include the Number of Terms served previously and the Number of Terms (Squared) to represent the curvilinear-shaped relationship between seniority and legislative posts. Prior work suggests that MPs with no prior parliamentary experience and those with very long tenures are less likely to obtain leadership posts compared to MPs elected multiple times. (Smith, Reference Smith2018). We add a binary variable indicating whether a candidate was elected in their single-member district (SMD) or as a Zombie candidate (i.e., they lost their SMD but got elected through the party's PR list). Pekkanen et al. (Reference Pekkanen, Nyblade and Krauss2006) show that “zombie” candidates often secure different posts compared to those elected in SMDs. We also control for the Gender of an MP and whether an MP belongs to a family with at least one previously elected member (Dynasty). These control variables were extracted from the dataset provided by Reed and Smith (Reference Reed and Smith2018).
Additionally, we account for the Ideological Distance between each MP's manifesto and the average position across all MPs included in our analysis. In line with Catalinac (Reference Catalinac2018), we use the unsupervised Wordfish scaling method (Slapin and Proksch, Reference Slapin and Proksch2008; Proksch et al., Reference Proksch, Slapin and Thies2011) to position all manifestos on a single latent unidimensional scale. We then determine the absolute difference between the Wordfish position of each candidate's manifesto and the average position of all manifestos within the party. Higher values signify a larger divergence between an MP's ideological stance and the party's average position.
4.4 Data structure and modeling approach
Before discussing the results, we briefly outline our data structure and modeling approach. Every MP is represented eleven times per election in the dataset, with each observation constituting one of the eleven policy areas. The binary dependent variable indicates whether an MP obtained a legislative post. The dependent variable is assigned the value 1 for the policy area(s) in which an MP held a post in the legislative cycle and 0 for all policy areas without a legislative post. We created separate dependent variables for different types of legislative posts: Cabinet Post, Committee Post, Party Policy Division Post, Ministerial Post, and the combined measure Legislative Post (Combined). The unit of analysis is therefore the candidate-cycle-policy area. We account for heterogeneity across elections, geographic regions, and policy areas through election fixed effects, region fixed effects, and policy area fixed effects. We cluster standard errors on the manifesto level (Bergé, Reference Bergé2018). As demonstrated in SI Section E.4, our results are highly consistent across model specifications and various sets of fixed effects.
5. Results
5.1 Descriptive evidence
We begin the results section by describing issue salience in Japanese candidate manifestos. Figure 2 displays the average salience in each election campaign and policy area, along with 95 percent bootstrap confidence intervals. Upon aggregating these averages across the five elections, Health, Labor, and Welfare is the most prevalent area with 23.5 percent of all policy-related statements. The focus on Health, Labor, and Welfare reached its peak in 2009. During the election campaign in 2009, the opposition party DPJ critiqued the ruling LDP's concentration on public projects and advocated for welfare expansion. Over 40 percent of policy-related sentences by DPJ candidates in 2009 pertained to health, labor, and welfare policies. Cabinet, encompassing policies like public safety, regional revitalization, gender equality, or disaster management, is the second most salient category (18.1 percent), followed by Education, Culture, Sports, Science, and Technology (10.7 percent). Security (3.4 percent) and the Environment (2.4 percent) represent the least salient issues.
Figure 2. Issue salience in candidate manifestos. Horizontal bars represent 95 percent bootstrap confidence intervals for each average. The dashed vertical lines and the numbers in the top-right corner of each box show the average issue salience across the five elections.
5.2 Regression analysis
Having described variation in issue emphasis across elections, we turn to testing our theoretical expectations. We run separate logistic regression models for the different types of legislative posts. Model 1 of Table 1 uses Legislative Post (Combined) as the dependent variable, measuring whether a legislator obtained at least one of the following: a Committee, Policy Division, or Cabinet Post. Models 2–5 represent separate models for Cabinet Post, Committee Post, Party Policy Division Post, and Ministerial Post. Across all models, we observe positive, sizeable, and statistically significant coefficients for Manifesto Salience. The log odds range from 2.19 (Model 2: Cabinet Post) to 2.92 (Model 4: Party Policy Division Post). Higher issue emphasis during campaigns increases the probability of obtaining a leadership post in the same area.
Table 1. Predicting legislative leadership posts in a policy area
The table reports log odds coefficients from logistic regression models. Standard errors (in parentheses) are clustered on the manifesto level.
Note: *** p < 0.001; ** p < 0.01; * p < 0.05.
We visualize the substantive effect sizes for the five types of posts to ease the interpretability of the coefficients (Arel-Bundock, Reference Arel-Bundock2023). Figure 3 depicts the predicted probabilities of obtaining a post conditional on the full range of Manifesto Salience. Higher issue emphasis increases the probability of obtaining a post in the same area for all measures of legislative leadership. The predicted probability of obtaining at least one leadership post in a given policy area (left-hand panel of Figure 3) is 0.22 (95 percent CI: 0.18, 0.26) when the respective issue is not mentioned at all. The predicted probability increases to 0.48 (95 percent CI: 0.39, 0.56) if a candidate devotes 50 percent of all policy-related statements to the policy area. The predicted probability of obtaining at least one post increases to 0.74 (95 percent CI: 0.62, 0.83) if the entire policy-related manifesto content was devoted to this policy area. The changes in predicted probabilities are also statistically significant across all posts, but less pronounced for Ministerial Post, mainly due to the limited availability of ministerial posts (see also Figure A1).
Figure 3. Predicted probabilities of obtaining legislative leadership posts conditional on the salience of the same policy area in candidate manifestos. Plot shows predicted probabilities based on Models 1–5 in Table 1. The remaining variables are held constant at their respective mean or modal values. Gray areas indicate 95 percent confidence intervals. The small vertical lines display the observed values of Manifesto Salience.
Recall that Hypothesis 2 posits a higher congruence between campaign communication and leadership posts in distributive policy areas than in public goods or high policy areas. We predict obtaining at least one legislative post [Legislative Post (Combined)] conditionally on an interaction effect of the three aggregated categories of policy areas (Distributive, High Policy, and Public Goods areas) with Manifesto Salience. We estimate predicted probabilities based on this interaction term, following the simulation-based approach suggested by King et al. (Reference King, Tomz and Wittenberg2000). The predicted probabilities in Figure 4 illustrate positive relationships between manifesto salience and legislative leadership posts. The relationship seems to be strongest for distributive policy areas, followed by public goods and high policy areas.
Figure 4. Predicted probabilities of obtaining a legislative leadership post, conditionally on the interaction effect between manifesto salience and the three broad issue areas, based on Model 1 in Table A7. The remaining variables are held constant at their respective mean or modal value. Gray areas indicate 95 percent confidence intervals. The small vertical lines display the observed values of Manifesto Salience.
To test whether these differences between broad issue areas are statistically significant, we simulate first differences – which is the difference between two predicted probabilities – across various values of Manifesto Salience (Greifer et al., Reference Greifer, Worthington, Iacus and King2023; Radean, Reference Radean2023). Figure A8 reveals that the predicted probabilities for distributive policy areas are significantly higher than for public goods and high policy areas across almost the entire range of manifesto salience. Across the ten scenarios of manifesto salience, the difference in predicted probabilities between distributive and public goods policy areas is 0.21; the difference between distributive and high policy areas is slightly lower (0.15). The first differences between high policy areas and public goods areas are smallest (0.06) and statistically insignificant across most values of manifesto salience. These results provide support for Hypothesis 2. The congruence between campaign communication and leadership posts is most pronounced for distributive policy areas.
Having established differences between issue areas, we assess the relationships for each of the eleven policy areas. We report the coefficients of Manifesto Salience on obtaining a post in this policy area based on separate regression models for each policy area. We report these coefficients for models predicting Legislative Post (Combined), Committee Posts, Cabinet Posts, and Party Policy Division Posts.Footnote 6 Figure 5 shows the log odds coefficients and 95 percent confidence intervals for Manifesto Salience for 44 regression models (one model per policy area, and separate models for the four types of posts).
Figure 5. Coefficient estimates and 95 percent confidence intervals of Manifesto Salience based on separate logistic regression models for each policy area. Standard errors are clustered on the manifesto level.
We observe positive and statistically significant coefficients for most policy areas: a higher emphasis on a policy area increases the log odds of getting a post in the same area. Yet, interesting differences emerge across policy areas. For our main outcome, Legislative Post (Combined), the coefficients of Manifesto Salience are sizable and significant for all distributive and public goods policy areas, but we observe more heterogeneity in the five High Policy areas. While the effect sizes for Foreign Affairs and Security are large and significant, the coefficients of Manifesto Salience for regression models focusing on Internal Affairs and Communications, Cabinet, as well as Financial Affairs are smaller and often fail to reach conventional levels of statistical significance.
To further investigate differences across issues, we apply an alternative measure of the importance of policy areas. Instead of running the models for different types of policy areas or conducting separate models for each policy area, we rely on a measure of Portfolio Importance derived from the expert surveys conducted at each election. A negative interaction coefficient between Manifesto Salience and Portfolio Importance implies that obtaining a post becomes less likely if a politician emphasizes a more important policy area. Figure 6 depicts the predicted values of obtaining a legislative post based on the interaction term between Manifesto Salience and Portfolio Importance in Model 1 of Table 2. We set Manifesto Salience to four values (0.1, 0.25, 0.5, 0.75) and predict the probability of obtaining a post conditional on the policy area's perceived importance. Figure 6 offers three insights: First, obtaining a post is unlikely across the entire range of policy importance when an MP rarely mentions the policy area. Second, the probability of success increases when MPs address a less important policy category more extensively in their manifesto. Third, for the most important policy areas (indicated through higher values on the horizontal axis), the predicted probabilities remain at very similar and consistently lower levels across the four scenarios of Manifesto Salience. MPs who avoid a policy area in their manifesto are less likely to secure a post, regardless of the policy area's perceived importance. However, MPs improve their chances of obtaining legislative posts by focusing more on less important policies in their manifestos. For key policy areas, the likelihood of success remains consistently low, irrespective of the space devoted to a policy area.
Figure 6. Predicted probability of obtaining a post (Combined Measure) conditional on the interaction effect between portfolio importance and varying levels of manifesto salience. The remaining variables are held constant at their respective mean or modal value. Predicted probabilities are based on Model 1 of Table 2. The small vertical lines display the observed values of Portfolio Importance.
Table 2. Predicting legislative leadership posts in a policy area conditional on the interaction effect between manifesto salience and perceived portfolio importance
The table reports log odds coefficients from logistic regression models. Standard errors (in parentheses) are clustered on the manifesto level.
Note: ***p < 0.001; **p < 0.01; *p < 0.05.
5.3 Robustness tests
We conducted a battery of additional analyses to test the robustness of our findings and further explore variation across issue areas. First, we run separate models for alternative categories of broad issue areas. We split the High Policy areas into High Policy (Domestic), consisting of Internal Affairs and Communications, Cabinet, and Financial Affairs, and High Policy (Foreign), comprising Foreign Affairs and Security. Results corroborate the findings from Figure 5: the relationship between campaign communication and leadership posts exists in foreign, but not domestic high policy areas (SI Section E.1). Second, we run jackknife-style regression models, which exclude one policy area at a time. The robustness test reveals that no single area drives our aggregated results. The relationship is even stronger when excluding Health, Labor, and Welfare from the analysis (SI Section E.2). Third, we run separate models for each electoral cycle. SI Section E.3 demonstrates the consistent presence of a positive relationship between manifesto salience and the allocation of legislative posts in all five elections. Fourth, we test the robustness of our findings with different model specifications. Neither the inclusion/exclusion of control variables nor different fixed effects specifications drive the observed relationship (SI Section E.4). Fifth, we run ordered logistic regression models with the count of posts in a policy area as our dependent variable instead of using a binary outcome. Results remain the same. A low focus on a policy area increases the probability of not obtaining any post, while the probability of obtaining one, two, or even three posts in an area increases with higher emphasis on the same area in candidate manifestos (SI Section E.5). Sixth, we use the count of sentences in each policy area as our independent variable, rather than the proportion of policy-related sentences, which has no effect on our results (SI Section E.6). These additional analyses provide strong support for the robustness of our findings.
5.4 Potential mechanisms
Although we observe a robust link between campaign communication and legislative priorities, the underlying mechanism driving this relationship remains uncertain. Consider the example of LDP MP Yasushi Kaneko during his third electoral term from 2005 to 2009. Kaneko had been elected since 2000, representing the Kumamoto 5th district, a rural district with farms and ports. In the September 2005 lower house election, he promised to support the development of local small and medium enterprises and farming and fishing communities and to promote the local economy, emphasizing infrastructure and transportation sectors in his election manifesto. Once elected, he became a Vice-Minister of Agriculture, Forestry, and Fisheries from November 2005 to September 2006, succeeded in preventing the suspension of dam construction in his district and securing the necessary budgetary allocations for the construction (Asahi Newspaper, 25 August 2006). Subsequently, he was promoted to Senior Vice-Minister of Land, Infrastructure, Transport, and Tourism, and achieved the improvement of the port in his district (Asahi Newspaper, 22 August 2009). The effective implementation of his policy priorities played a pivotal role in his re-election even in the 2009 election where the LDP reduced its seats from 300 to 119 and ultimately lost power.
This example raises a question: Did Kaneko become senior vice-minister due to his voters' interests and intrinsic motivations, or did he obtain the leadership posts because he held posts in the same area previously? While our observational study does not allow us to identify causal effects, we can investigate these potential mechanisms in two ways. First, we might expect that holding a leadership post in a given policy area increases the chance of regaining the post in the same area after the next election. If campaign communication does not play a role in obtaining leadership posts, the effect may disappear when controlling for previous legislative posts. Table A13 adds the lagged dependent variable, which is a binary indicator representing whether an MP held the post in the previous cycle. Including the lagged dependent variable decreases the number of observations by over 50 percent, as MPs who were elected for the first time or in 2003 cannot be considered in the analysis. The coefficient of Post in Previous Cycle has the expected positive, sizeable, and statistically significant coefficient, ranging from 1.71 (Legislative Post [Combined]) to 2.46 (Ministerial Post). Yet, even after controlling for legislative leadership in the previous cycle, the coefficients for Manifesto Salience remain positive and statistically significant, suggesting that prior experience and campaign communication matter for obtaining legislative leadership posts.
The second mechanism relates to an MP's political experience. If representatives' campaigns were irrelevant, we should not observe any relationship between manifesto salience and posts for politicians who never have been elected before. To test this possibility, we run a regression model for the subset of politicians who were elected for the first time (Model 1 of Table A14). We repeat this subsample analysis for politicians who were elected for the second time (Model 2), for the third time (Model 3), and four or more times (Model 4). While first-timers are less likely to obtain legislative posts than their more experienced colleagues (Smith, Reference Smith2018 and Figure A15), we still observe a positive and statistically significant coefficient of Manifesto Salience for MPs who won an election for the first time. The fact that even inexperienced politicians' emphasis in manifestos translates into legislative posts provides further evidence of a link between campaign communication and legislative priorities. At least some portion of a representatives' interest – measured through the emphasis in manifestos – comes before any appointment by the party leader.
6. Conclusion
The concept of promissory representation posits that campaign communication provides a truthful signal of politicians' policy priorities. We test this assumption empirically with novel datasets of issue emphasis in candidate manifestos and legislative leadership posts, covering five Japanese elections. Our results are encouraging. Communication in candidate manifestos predicts subsequent legislative priorities. Candidates who focus extensively on specific policy areas are more likely to obtain leadership posts in this area, confirming Sulkin's (Reference Sulkin2009) evidence from the U.S. House of Representatives. Various model specifications and measures of campaign communication confirm that politicians often translate the policies they emphasized during campaigns into legislative actions. In other words, voters can use these candidate manifestos to identify politicians' policy priorities. Campaigns link policy in the electoral and parliamentary areas.
Yet, several questions remain. The role of party leaders in assisting the congruence between individual politicians' campaign communication and legislative activities could be investigated in more detail. While we find that manifesto salience predicts leadership posts, the current set-up does not allow us to identify how candidates use their campaign communication to signal to party leaders that they will be suitable and motivated holders of leadership posts.
Another open question relates to the external validity of our findings. While Japan is an established representative democracy, candidate manifestos do not exist in most democracies. Future research could move beyond manifestos and assess individual-level issue emphasis in a variety of ways. We hope that our findings encourage further work on individual MP-level congruence for additional types of communication, contexts, and countries. For example, scholars could compare candidates' statements on social media (Barberá et al., Reference Barberá, Casas, Nagler, Egan, Bonneau, Jost and Tucker2019), campaign speeches, or election leaflets (Trumm et al., Reference Trumm, Milazzo and Duggan2023) with the content of legislative speech (Proksch and Slapin, Reference Proksch and Slapin2015), legislative posts (Bergman et al., Reference Bergman, Struthers, Shugart, Pekkanen and Krauss2022), or communication on social media (Castanho Silva and Proksch, Reference Castanho Silva and Proksch2022). Our finding that pledges constitute a large share of the content of manifestos opens up avenues for future research on candidates' pledge making and fulfillment, which has mainly been assessed on the party level (e.g., Thomson et al., Reference Thomson, Royed, Naurin, Artés, Costello, Ennser-Jedenastik, Ferguson, Kostadinova, Moury, Pétry and Praprotnik2017; Naurin et al., Reference Naurin, Royed and Thomson2019). What are the similarities and differences between pledges made by different candidates? How do these pledges of candidates relate to the pledges that their parties make in national programs (Proksch et al., Reference Proksch, Slapin and Thies2011)? And under which conditions are candidate pledges more likely to be fulfilled? We hope that our research design and results encourage future work on the relationship between campaign communication and legislative behavior.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/psrm.2024.11.
To obtain replication material for this article, https://doi.org/10.7910/DVN/8783YN
Acknowledgements
We thank Shaun Bowler, Frederik Hjorth, Sarah King, Joseph Lacey, Yen-Chieh Liao, Robin Rauner, Chiara Valsangiacomo, the editor and anonymous reviewers, and participants at the Annual Conference of the European Political Science Association in Prague, the Politics Research Colloquium at the University of Oxford, the workshop on Election Campaigns and Democratic Norms in Dublin, and the SPIRe Seminar Series at University College Dublin for their comments and improvement suggestions. We are grateful to Yutaka Shinada for sharing digitized candidate manifestos. We would like to thank Daisuke Hakiai, Keiko Higuchi, Momoho Kikukawa, Kana Miyazaki, Tsubasa Nishimura, and Ken Yoshioka for outstanding research assistance.
Financial support
This research was supported by the Grant-in-Aid for Scientific Research from the Japan Society for the Promotion of Science (research project number: 18K01413), the Research Fund from the Quantitative Methods for International Studies Program at Kobe University, and the UCD Ad Astra Start-Up Grant. Open access funding was provided by IReL.
Competing interests
None.
Open access
