It’s (Not) in The Reading: American Government Textbooks’ Limited Representation of Historically Marginalized Groups

Abstract

The Introduction to American Government course, and its textbook, is a nearly universal experience for students in American colleges and universities, but what exactly is being taught in this course? Do the textbooks used in this widely taught course accurately reflect the diversity of populations and experiences in the United States? More specifically, how do textbooks for Introduction to American Government cover historically marginalized groups, if at all? This article builds on previous work by analyzing the representation of individual historically marginalized groups to conduct index search and content analyses on traditionally published and openly licensed (i.e., open educational resources [OER]) textbooks. This study finds that American government textbooks include little coverage of any historically marginalized groups, and that OER textbooks are average in this respect, doing neither better nor worse than their traditionally published counterparts.

Although it is a foundational course in political science, the content of Introduction to American Government is seldom studied within the discipline. Nevertheless, it is important to look at the content of these textbooks because they are the first and possibly only exposure students have to the field of political science, and they represent to students what is deemed important in American government (Atchison 2017). Do the materials used in these courses reflect the diversity of populations and experiences in the United States?¹

ANALYSES OF REPRESENTATIONS OF HISTORICALLY MARGINALIZED GROUPS IN POLITICAL SCIENCE

The limited representation of historically marginalized groups has been examined across several subfields of political science. Tolley (2020) analyzed five popular introductory textbooks on Canadian government and found limited coverage of immigrants and minorities. Looking at 10 general introduction to political science textbooks, Atchison (2017) found a similar lack of coverage of women, indicating a systematic problem across the discipline, not only in American government textbooks. A comprehensive review (Knutson 2017, 536) of five American government textbooks, aimed at assisting instructors “who value different approaches” to select texts for course adoption, did not include coverage of historically marginalized groups as reviewing criteria, implying that it is not something Knutson (2017) believes instructors would be interested in when deciding on a textbook. In this, Knutson was following the general approach of the discipline of political science, as highlighted by Strach (2019, 12), who summarized how these issues are marginalized: “Political scientists consider such work to be about race and politics, or sexuality and politics, or gender and politics, not politics.”

Several studies analyze American government textbooks for representation of different races and ethnicities, as well as women and LGBTQ persons, establishing a baseline for representation in American government textbooks. Ashley and Jarrat-Ziemski (1999, 59) used an index keyword search to compare the amount of coverage of Native Americans to other minority groups, finding “that American government texts do a generally abysmal job when it comes to adequately dealing with Native Americans.” Novkov and Gossett (2007, 393) examined LGBTQ representation in 17 American government textbooks; they found that all of the texts included references to lesbians and gay men “almost universally in discussions of civil rights and/or equality,” bisexuals were rarely included, and transgender individuals were completely absent.

Modeling on Novkov and Gossett (2007), Wallace and Allen (2008) found that African Americans are the most represented of any racial/ethnic groups, but that most of the reviewed textbooks ignore the contributions of African Americans to American political development until the civil rights movement—even then, representations of this involvement are siloed in standalone sections or chapters on civil rights. Using Wallace and Allen (2008) and Novkov and Gossett (2007) as models, Monforti and McGlynn (2010) examined Latinx representation. Their findings concur with Wallace and Allen’s (2008), in both the limited amount of coverage and the exiling of that coverage to civil rights chapters. Takeda (2015) used an index-search–based approach on the most recent editions of the textbooks in Monforti and McGlynn (2010) for coverage of Asian American/Pacific Islanders and found that an extremely small number of pages included coverage of this group.

With regard to gender representation in American government textbooks, Olivo (2012) and Cassese, Bos, and Schneider (2014) came to a similar conclusion as the authors who studied race and ethnicity: “[c]ontent about women is scarce, marginalized, and stereotypical” Cassese, Bos, and Schneider (2014, 254).

Taken together, these studies indicate that coverage of historically marginalized groups is extremely limited in American government textbooks. By combining the search terms from these studies into one study, I provide a comprehensive view of representation (or lack thereof) of historically marginalized groups in American government textbooks.

TEXTBOOK COST AND OPEN EDUCATIONAL RESOURCES

It is impossible to discuss textbooks without discussing their exponentially rising cost. Textbook costs have been steadily rising since the late 1990s, averaging a 6% increase per year through 2016 (Perry 2019). The high cost of textbooks has been linked to negative student outcomes, such as not purchasing required textbooks, earning poor grades, and dropping courses (“2018 Student Textbook and Course Materials Survey” 2019, 3). One response to the increasing costs of textbooks has been the creation and adoption of open educational resources (OER). OER are “teaching, learning, and research materials in any medium, digital or otherwise, that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation, and redistribution by others with no or limited restrictions” (Paris OER Declaration 2012). Although OER have been in use for nearly two decades (Cronin 2018) and their efficacy has been studied in several courses from various institutions and perspectives,² they are even less studied than textbooks by the journals published by the American Political Science Association. There are two OER articles in Journal of Political Science Education, one of which I wrote, and none in the other three journals.

CULTURALLY RESPONSIVE TEACHING AND OER

There is significant research that establishes the need for culturally responsive teaching in primary and secondary education. Ladson-Billings (2009) demonstrated that teaching that ignores African American students’ culture is teaching that significantly decreases those students’ chances for academic success. Gay (2000) demonstrated how students of color are continually shortchanged by pedagogies, curricula, and textbooks that ignore their cultures and identities. She called for culturally responsive teaching, including revising and supplementing teaching materials and textbooks, to improve educational outcomes for students of color. Larke (2018, 38) argued that culturally responsive teaching practices should be adopted by instructors of higher education because it leads to more equitable outcomes for all students, especially as higher-education student bodies become “more and more culturally, linguistically, economically, and ethnically diverse.” Because their low costs make them accessible to all students regardless of their income, and because they are licensed for revision and remixing, OER have the potential to address both equity and inclusion concerns. It is important to note, however, that OER are not necessarily automatically more culturally responsive.

HYPOTHESIS

I expect that current American government textbooks will have extremely limited coverage of historically marginalized groups whereas OER textbooks will have more and better coverage of historically marginalized groups (operationalized as African Americans, Latino/a Americans, Asian Americans, Native Americans, women, and LGBTQ persons) than their traditionally published counterparts for several reasons. Previous studies of representations of historically marginalized groups in American government textbooks reveal little coverage of these groups, making it likely that current traditionally published books will continue that trend, thereby making it a low threshold for OER books to exceed. Additionally, the content in OER textbooks has been developed more recently rather than being revised editions of materials that are a decade or more older. Finally, OER textbooks are part of the open education movement, which seeks to address more than just the inequality-of-access issues due to the rising costs of textbooks. A major benefit touted by OER advocates is the ability to produce more up-to-date, representative, and culturally responsive materials.

METHODOLOGY

To explore whether and how introductory American government textbooks cover historically marginalized groups, I adapted the methods of previous studies, conducting both an index search and a full-text content analysis. To obtain a wide variety of texts, I used three different means to select traditionally published textbooks for the index search. I selected four of the most recent editions of the previous studies’ highest ranked textbooks, three textbooks used in American government syllabi in APSA’s syllabus bank, and three textbooks from Amazon’s Top 100 government textbooks list, as well as one OER textbook.³ I conducted an index search using a list of terms combined from the articles covering individual historically marginalized groups, to which I added three additional search terms for essential US Supreme Court decisions that relate to LGBTQ rights that did not exist at the time of the Novkov and Gossett (2007) paper: United States vs. Windsor, Hollingsworth vs. Perry, and Obergefell vs. Hodges. This resulted in a search list of 205 unique terms,⁴ which then were coded according to which historically marginalized group the term referred: General (23 words, for terms that were not obviously applicable to a single group); African American (6); African American and Women (2); Asian American/Pacific Islander (27); Latinx (47); Latinx and Native American (1); Latinx and Women (4); Native American (3); LGBTQ (18); and Women (74). Individual listing pages were counted for each entry on the list.⁵

I also conducted a content analysis of entire texts by collecting PDF files of 13 textbooks for analysis in Wordstat: four free digital textbooks⁶ and nine traditionally published textbooks. A categorization dictionary derived from the index search words was created to see how frequently each historically marginalized group was mentioned throughout the textbooks.⁷

INDEX SEARCH RESULTS

Across the 11 textbooks analyzed, 3,544 pages were found in the indices for 131 search terms. This is a small number compared to the total 5,735 non-index and non-appendix pages of all 11 books.⁸ Not only were few pages indexed, but also only 61% of the 205 search terms (131) had at least one page indexed in at least one book; this means that one third of the search words were not present in the indices of any of the 11 books analyzed. Fifteen search words were indexed on only one page in one book. Only 65 search words were indexed on 10 or more pages across the 11 books analyzed, and only five search words were indexed in all 11 books: Voting Rights Act, Roe vs. Wade, affirmative action, Elena Kagan, and the Defense of Marriage Act. Limited representation of most marginalized groups was consistent across all textbooks reviewed, with McClain and Tauber (2018) as a noticeable outlier (figure 1). The average number of search-word pages per book was 322, with seven of the 11 books less than the average. Figure 1 also shows that the representation of different historically marginalized groups is not evenly distributed. General search words that were not specific to any one marginalized group were the most indexed; words related to women came in second. The highest number of indexed pages in each book was largely from the General category, with other groups receiving much less coverage in every book and little coverage overall.

Figure 1 Distribution of Index Pages Found for Each Historically Marginalized Group

Figure 2 Historically Marginalized Group Word Frequencies in 13 American Government Textbooks

The total number of indexed pages is somewhat deceptive because 3,544 reflects numerous duplicate pages; that is, the same textbook page might be indexed for several different search words. Table 1 shows the number of deduplicated pages that had at least one search word on a page. When the 2,001 duplicate pages were removed, only 1,543 unique pages across all 11 books had at least one search word, representing 26.9% of the substantive pages in all 11 books. The textbooks varied widely in the percentage of pages that had at least one indexed search word. The Openstax book scored lowest with 10%, whereas the McClain and Tauber (2018) percentage of 60.5% is almost twice the next highest percentage from Welch et al. (2014), which is still far above the average.

Table 1 Indexed Search Words, Deduplicated Pages, and Pages in Civil Rights Chapters in 11 American Government Textbooks

TEXTBOOKS ANALYZED

The following is the list of textbooks analyzed:

1. 1. Krutz et al. 2019

2. 2. Welch et al. 2014

3. 3. Gaddie and Dye 2018

4. 4. O’Connor and Sabato 2018

5. 5. Schmidt, Shelley, and Bardes 2018

6. 6. Barbour and Wright 2015

7. 7. Ginsberg et al. 2017

8. 8. Kollman 2015

9. 9. McClain and Tauber 2018

10. 10. Morone and Kersh 2018

11. 11. Patterson 2015

The frequency of search words occurring on the same pages indicates a clustering of representation of historically marginalized groups rather than these groups being included throughout the text. Tolley (2020) found that Canadian political science textbooks often silo their representations of immigrants and minorities in diversity-specific chapters, presenting oppression of these groups as historical artefacts, not contemporary issues. Novkov and Gossett (2007), Wallace and Allen (2008), Takeda (2015), and Monforti and McGlinn (2010) found similar ghettoization. To test whether this is still the case in American government textbooks, deduplicated pages were coded for the chapter in which they appeared in each book and whether that chapter was the civil rights chapter. All but one of the 11 textbooks siloed more than 20% of their inclusion of historically marginalized groups into chapters on civil rights (see table 1). The average across all books was 26.8% of indexed mentions of historically marginalized groups that were siloed in civil rights chapters. Moreover, 18 chapters across six books did not contain a single indexed reference to a historically marginalized group.

All but one of the 11 textbooks siloed more than 20% of their inclusion of historically marginalized groups into chapters on civil rights…

Based on the index analysis, it appears that American government textbooks do not devote much coverage to historically marginalized groups. The book with by far the highest number of indexed pages, the most deduplicated pages, and the lowest percentage of siloed pages was McLain and Tauber’s (2018) American Government in Black and White: Diversity and Democracy. This is perhaps unsurprising because it also is the only book with an explicit focus on racial and ethnic equality as a framing principle. The OER textbook is not more representative than its traditionally published counterparts, scoring low on most counts of the index analysis.

But is the index analysis alone a sufficient measure of representation of historically measured groups? The Openstax index is shorter than its traditionally published counterparts, which could explain some of the reason that it scores so far behind them. Index searching is subject to specific word order, which might leave some pages undercounted. Finally, as more students access digital versions of their textbooks, readers are less likely to use the index at all. It is therefore necessary to examine the content of the textbooks to accurately determine the coverage that historically marginalized groups receive.

FULL-TEXT ANALYSIS RESULTS

Across all books, words referring to historically marginalized groups came up rarely: of 3,741,902 total words, only 31,340, or 0.84%, were in the categorization dictionary. Moreover, only 7.2% of sentences and 8.9% of paragraphs included at least one word from the categorization dictionary. Differences in length might explain the variation in the textbooks’ use of words; however, when frequency of categorized words per 10,000 words was calculated, none of the textbooks mentioned words related to historically marginalized groups all that frequently (table 2). The infrequent use of these words across all of the textbooks analyzed here indicates that political science as a discipline does not consistently consider these words essential to the study of American government. Again, McClain and Tauber (2018) and Welch et al. (2014) come in first for having used the most categorized words and having the highest rate of categorized words, with the remaining textbooks significantly farther behind. The OER textbooks were neither the highest nor the lowest ranked, meaning that they were neither much better nor much worse than the average of American government textbooks. By also rather seldomly using words about historically marginalized groups, OER fit right in with the bulk of their traditionally published American government peers.

Across all books, words referring to historically marginalized groups came up rarely: of 3,741,902 total words, only 31,340, or 0.84%, were in the categorization dictionary.

By also rather seldomly using words about historically marginalized groups, OER fit right in with the bulk of their traditionally published American government peers.

Table 2 Number of Historically Marginalized Group Words and Rate of Categorized Words per 10,000 Words in 13 American Government Textbooks

Abbreviations Used for Textbooks:

McClain and Tauber–AGBW: McClain and Tauber 2018

Welch et al.–UAG: Welch et al. 2012

OER–21st Century AG: 21st Century in American Government 2012

OER–AGP in Info Age: American Government and Politics in the Information Age 2016

OER–Lenz Holman: Lenz and Holman 2013

OER–Openstax 2e: Krutz et al. 2019

O’Connor et al.–AGRR: O’Connor, Sabato, and Yanus 2016

Schmidt et al.–AGP: Schmidt, Shelley, and Bardes 2018

Barbour and Wright–KTR: Barbour and Wright 2015

Ginsberg et al.–WTP: Ginsberg et al. 2017

Kollman–APS: Kollman 2015

Morone and Kersh–BtP: Morone and Kersh 2018

Patterson–WTP: Patterson 2015

Similar to the index results, word use was not evenly distributed across historically marginalized groups: words from the General and Women categories received the largest amount of coverage, followed by African American words.

CONCLUSION

It is clear that the American government textbooks studied herein have rather little coverage of historically marginalized groups, whether analyzing indices or in-text frequency. It also is clear that not all groups receive the same amount of coverage, with General terms and Women receiving far more coverage than African Americans, who are covered more than Latinx Americans, LGBTQ Americans, and Asian American/Pacific Islanders.

The reduction of profit motivation is one reason to believe that OER textbooks might handle representation of historically marginalized groups better than their traditionally published counterparts. Yet, based on this analysis, OER textbooks do not perform any better (or any worse) than their traditionally published counterparts when it comes to representation of historically marginalized groups. Elimination of the profit motivation is not sufficient to ensure coverage of historically marginalized groups in American government textbooks.

OER textbooks may have an advantage in their open licensing; that is, instructors are free to revise and republish their own version of openly licensed textbooks. Instructors often supplement textbooks in their courses with articles or summaries, but they are not permitted to rewrite and reissue the text, which limits their ability to share their corrections. OER materials officially permit and often encourage these revisions for faculty who want to make them. In fact, Openstax recently announced that it will be releasing Google Doc versions of all of its textbooks, which will make editing their books even easier (Williamson 2020).

Similar scores among so many different textbooks, both OER and traditionally published, indicates a broad consensus in the way historically marginalized groups are covered in introductory American government textbooks: in a very limited way. The outlier, the McClain and Tauber (2018) textbook, is much more inclusive of historically marginalized groups only because it explicitly intends to be so. Lacking that special intention, the other 12 textbooks examined here default to limited inclusion of historically marginalized groups.

Further research is called for on this topic. Examining the images that textbooks use, as Clawson and Kegler (2000) did, is a fruitful direction because the visual component of textbooks is essential to students’ experience of their learning materials. Examination of representations of other marginalized populations, such as people with disabilities or members of religious minority groups, is also warranted, although it fell outside the scope of this project. Finally, in the process of doing this research, it became evident just how much these 205 search terms themselves exclude. More detailed studies that begin with the contents of textbooks and their indexes instead of a list of search terms would be fascinating to see.