Further Insights on Fake-Barn Cases and Intuition Variation

Abstract

Studies in experimental philosophy claim to document intuition variation. Some studies focus on demographic group-variation; Colaço et al., for example, claim that age generates intuition variation regarding knowledge attribution in a fake-barn scenario. Other studies claim to show intuition variation when comparing the intuition of philosophers to that of non-philosophers. The main focus has been on documenting intuition variation rather than uncovering what underlying factor(s) may prompt such a phenomenon. We explore a number of suggested explanatory hypotheses put forth by Colaço et al., as well as an attempt to test Sosa's claim that intuition variance is a result of people ‘filling in the details’ of a thought experiment differently from one another. We show (i) that people respond consistently across conditions aimed at ‘filling in the details’ of thought experiments, (ii) that risk attitude does not seem relevant to knowledge ascription, (iii) that people's knowledge ascriptions do not vary due to views about defeasibility of knowledge. Yet, (iv) we find no grounds to reject that a large proportion of people appear to adhere to so-called subjectivism about knowledge, which may explain why they generally have intuitions about the fake-barn scenario that vary from those of philosophers.

1. Introduction

The type of traditional philosophy sometimes labelled armchair philosophy is often characterized as making use of a method that relies on intuitions. Although intuition-based methodology is not assumed to be a methodology that guarantees the correctness of its results, it is assumed to be sufficiently reliable to be trusted. However, a number of studies in experimental philosophy purportedly show that philosophical intuitions vary according to factors such as cultural and educational background (Weinberg et al. 2001; Machery et al. 2004), socio-economic status (Haidt et al. 1993; Nichols et al. 2003), or emotions.¹ Call this phenomenon intuition variation. Intuition variation is problematic in so far as the evidential value of intuition is considered to be linked to its universality. If demographic factors really do have an impact on intuitions in the way suggested, this is likely to put the reliability of intuitions into question.

So far, the empirical part of the intuition debate has primarily operated according to a somewhat straightforward recipe. Subjects are presented with vignettes structurally resembling paradigmatic philosophical thought experiments (Gettier-type cases for example), then typically they are asked simple binary questions (such as: ‘Is the situation as described by the vignette an instance of knowledge?’). Results are then subjected to (statistical) analysis, in order to be able to show: ‘‘intuition varies according to x variable(s)’. The focus has mainly been on various demographic factors such as age, gender or cultural background. However, these factors have often been studied in isolation, which leads to a worry that interacting or further confounding variables are being ignored. Furthermore, rarely do these studies go beyond merely identifying that a given demographic factor varies, to examining why it influences intuition. For example, although Weinberg et al. (2001) find culture-specific intuition variation, they offer no explanatory hypothesis (or empirical testing) of why that is. Likewise, Buckwalter and Stich (2014) report having found gender-specific intuition variation. Although they hypothesize how such intuition variation might contribute to the underrepresentation of women in philosophy, no explanation (or even hypothesizing) is offered regarding what it is inherently about being a man or a woman that gives rise to such variation. As a result, the debate regarding these empirical results remains at a somewhat superficial level – there is no attempt to dig deeper and, consequently, to inform us of what precisely the decisive factor(s) in a given intuition variation might be.

A welcome departure from this norm is Colaço et al.'s (2014) paper on the fake-barn intuition, in which the authors present data showing that intuition varies according to age. More specifically, they find that “the intuition that fake-barn cases do count as knowledge is negatively correlated with age; older participants are less likely than younger participants to attribute knowledge in fake-barn cases” (Colaço et al. 2014: 199). Colaço et al. (2014) speculate whether their findings regarding age might be driven by one of the following three factors. First, they speculate whether risk aversion might play a role. On the back of results indicating a relationship between risk attitudes and age (Bonsang and Dohmen 2012), Colaço et al. (2014: 207) suggest that “more risk averse people may judge that more is at stake in a fake-barn case than less risk averse people”. If “knowledge-attribution is stake-sensitive, then older people should be less likely to attribute knowledge in fake-barn cases” (Colaço et al. 2014: 207). Second, they hypothesize whether previous experiences with non-knowledge beliefs might play a role. Perhaps older people (naturally) have had “more experiences of merely apparent knowledge” (Colaço et al. 2014: 207) than younger people and so are more likely to take such accumulated experiences into consideration when contemplating fake-barn cases. We take this to be a question about whether people's views about defeasibility of knowledge – that you can know p at t₁ without knowing p at t₂ – play a role in knowledge attribution in fake-barn scenarios. Third, the authors speculate whether “younger people may be more likely to tacitly endorse some form of epistemic subjectivism than older people” (Colaço et al. 2014: 207). This latter suggestion they considered to be the least plausible option. Later studies have attempted to replicate the correlation between age and knowledge attribution in scenarios similar to a fake-barn, but have not succeeded (Seyedsayamdost 2014; Turri Unpublished²; Knobe Unpublished³). Disregarding whether age is a significant factor or not, it is still an open question whether the three factors that Colaço et al. (2014) put forward might be driving intuition variance.

2. Explaining intuition variation and the expertise defence

Our study took as its starting point a replication of Colaço et al.'s (2014) experimental set-up. While we were unable to replicate Colaço et al.'s (2014) result regarding age, we further pursued Colaço et al.'s (2014) three suggested explanatory features of their results (risk aversion; previous experience; epistemic subjectivism). Our assumption was that the original suggested hypotheses may in fact explain – independently of the age variable – why people judge that the protagonist in the fake-barn case knows. For example, if we consider risk attitude to vary among people, one should expect there to be a correlation between risk willingness and knowledge attributions in certain environments where knowledge appears to be volatile or lucky. Rather than to attempt to explain why people of a certain age are more or less likely to attribute knowledge due to an expected risk profile, we directly analysed people's attitudes towards risk and how this correlates with how they make judgements in the fake-barn scenarios.

In addition to testing the three hypotheses suggested by Colaço et al. (2014), we also wanted to pursue an explanation for why differences in responses are registered between participants with no philosophical training and philosophers.⁴ In particular we wanted to test one way of understanding what the so called ‘expertise defence’ comes to. The expertise defence is an explanation-strategy according to which we should distinguish between intuitions as elicited by lay people (i.e. the majority of subjects in the relevant studies) and intuitions as elicited by philosophers (for example Ludwig 2007; Williamson 2007; Horvath 2010; Devitt 2011).

This strategy is meant to disassociate the question of the reliability of philosophers’ intuition from that of people in general.

The expertise defence is built on the assumption that ordinary participants are subject to some degree of shortcoming (due to their lack of philosophical training) and one way of spelling out what this kind of shortcoming could amount to may be understood in the following way. Sosa (2009: 107) notes how, when reading fiction, we “import a great deal that is not explicit in the text”. Although of course we “follow the author's lead in our own imaginative construction” (Sosa 2009: 107), we cannot help but import a great deal of “normally presupposed” detail – details that may plausibly differ, given how we as individuals differ (specific traits, demographic variables etc.). Sosa speculates that these ‘differently-filled-in-details’ may result in different judgements as the result of thought experimentation, i.e. intuition variation.⁵ Maybe, the thought goes, the intuitions of philosophers are different from those of lay people simply because philosophers ‘fill in the details’ consistently differently from the way in which lay people ‘fill in the details’ – and differently ‘filled-in-details’ supposedly generate different intuitions. As called for by Nado (2014), we need more clarity about what we actually mean by expertise when we talk about philosophers’ alleged expertise, in order to test it. One way to understand the expertise defence, then, is to suggest that the way in which philosophers ‘fill in’ and subsequently judge in regard to a given thought experiment is the correct one. If we wanted to test this idea, we should attempt to assist subjects in their ‘filling in’ and level out the difference between trained philosophers and non-experts. After such levelling has taken place, we should expect a result where lay responses to thought experiments would be in alignment with those of philosophers. There are different ways in which such assistance could be attempted.

Turri (2013), for example, focuses on highlighting the structure of certain thought experiments. Turri speculates that “laypeople who answer that the Gettier subject knows aren't competently enough assessing the case” (2013: 2). More specifically, the philosopher becomes aware and understands the significance of certain features in the cases that the untrained layperson tends to overlook. Therefore, Turri speculates, “if we effectively guide participants to notice and assign proper weight to those same features, then their responses will be similar to philosophers’” (2013: 2). That is, the responses of laypeople will shift towards the result that the protagonist in a Gettier case does not know. Turri believes that drawing attention to the structure is key in obtaining just this.

Turri's ‘guiding of the lay person’ consists in presenting a number of Gettier-style cases divided into three distinct sections. Turri understands – mirroring Zagzebski (1996) – Gettier cases as having a distinctive tripartite structure.⁶ By dividing the case into three different sections and instructing subjects to track the truth value of the primary proposition through these stages, subjects are guided, or so Turri claims, “to notice the intersection of evidence, truth, and luck, and highlighting that the bad luck's source differs conspicuously from the good luck's source” (2013: 5). Result-wise Turri (2013: 15) reports that when probing subjects this way (“across very different cultures, male and female, young and old”), they “competently assess Gettier cases”, that is, they adhere to philosophical orthodoxy of withholding knowledge from the Gettier protagonist.

But there might be more direct ways of assisting and ‘effectively guiding’ subjects – i.e. there might be more straightforward ways in which we can help subjects by ‘filling in the details’. There is a substantial literature on how pictorial representations yield far better learning results when accompanying textual representation compared with presenting textual representation by itself. According to one study, for example, people following directions with text and illustrations do substantially better than people following directions without illustrations (Levie and Lentz 1982). Another study found, when studying the comprehension rates of medicine labels, a 70% rate of understanding for labels with text only and a 95% rate of understanding for labels with text and pictures (Dowse and Ehlers 2005).

Indeed, the didactic significance of illustrations is generally recognized. Human beings are what we might term ‘visual learners’ – 50% of our brain is involved in visual processing, and we can come to have a sense of a visual setting in less than 1/10th of a second (Semetko and Scammell 2012). An illustration can function to simplify complex information, as well as to concretise abstract information, by conveying spatial, temporal and functional relationships in a way that is simpler, yet more complete than those conveyed merely by linguistic descriptions.

The awareness of such results led us to formulate the following hypothesis: by providing lay people with a pictorial representation of a well-known thought experiment (see Figure 1) (a so-called fake-barn case) we would – if Sosa's ideas were correct – notice an increase in the number of subjects withholding knowledge attribution judgements in a fake-barn situation – because, we hypothesized – a visual representation of a thought experiment would provide even more ‘effective guiding’ towards becoming aware and understanding the significance of important features in the case.

Figure 1. Cited from Bergenholtz et al. Forthcoming.

Our results show no support for intuitions varying across different age or educational groups, while we do find a clear gender effect, with women being significantly more likely to attribute knowledge (72.2% vs. 59.8). Our tests of Colaço et al.'s (2014) three suggested hypotheses reveal, that (i) willingness to engage in risky behaviour is not correlated with attributing knowledge. Furthermore, (ii) views about defeasibility of knowledge do not appear to be related to knowledge attribution, while (iii) what we categorize as epistemic subjectivism is significantly correlated with the likelihood of attributing knowledge to the protagonist.

Our results, in effect, call for a more nuanced theoretical approach in the area of experimental philosophy investigating intuition variation in Gettier-type cases. Specifically, they provide reason to believe that different people may attribute knowledge in fake-barn cases for different reasons. This calls for a multi-explanation theoretical approach that suggests that multiple factors underpin the alleged observable intuition variation. Whether this applies only to fake-barn cases, or whether it may be a more general trend applicable to a variety of intuition variation cases across different philosophical domain needs to be investigated further.

3. Methods

We engaged in two rounds of data collection in order to test our hypotheses. In the first round we either exposed participants to a regular text-based presentation of the fake-barn scenario or added a pictorial representation to the text-based presentation (cf. Figure 1). Since the pictorial representation substantially reduced the number of people we had to exclude due to comprehension failures (Bergenholtz et al. Forthcoming), we only relied on the picture in a second round of data collection. We added a second round to include further questions enabling a more fine-grained analysis of the responses. See Appendix 1 for a list of questions. Relying on the Amazon Mechanical Turk (MTurk) platform we collected responses from 428 participants in total (female: 47.7%, mean age: 37.8, SD: 10.9, age range 19–70). Repeat participation in both rounds was prevented via a systematic check of all Mturk IDs. We excluded people with a philosophy degree and respondents who reported having heard about Gettier cases before, leading to a sample of 395.⁷ In the following we rely on the combined sample (text only and picture) when assessing demographic variables, risk and the comparison between picture and text (sections 4–4.2), while we rely on the picture-only based sample when testing our more fine-grained hypotheses (sections 4.3–4.4).

Participants were randomly assigned to one of two conditions (the textual representation condition and the textual & pictorial representation condition) in a between-subjects experimental design. Following the experimental set-up of Colaço et al. (2014), participants in the textual representation condition read a vignette structurally very similar to Goldman's fake-barn case, only – again following Colaço et al. (2014) – the barns were replaced by houses:

Gerald is driving through the countryside with his young son Andrew. Along the way he sees numerous objects and points them out to his son. ‘That's a cow, Andrew,’ Gerald says, ‘and that over there is a house where farmers live.’ Gerald has no doubt about what the objects are. What Gerald and Andrew do not realize is the area they are driving through was recently hit by a very serious tornado. This tornado did not harm any of the animals, but did destroy most buildings. In an effort to maintain the rural area's tourist industry, local townspeople built house façades in the place of destroyed houses. These façades look exactly like real houses from the road, but are only for looks and cannot be used as actual housing. Though he has only recently entered the tornado-ravaged area, Gerald has already encountered a large number of house façades. However, when he tells Andrew ‘That's a house,’ the object he sees and points at is a real house that has survived the tornado.

For the participants in the textual & pictorial representation condition, the vignette was accompanied by a picture illustrating the scenario (see Figure 1).

On the basis of Sosa's (2009) speculations, that the ‘differently filled in details’ should result in different judgments as the result of thought experimentation, we should expect to see intuition variation. Thus, we expected to see a change in the number of people who state that Gerald knows that he saw a house, between the text-only condition and the text & picture condition.

4. Results

We find that respondents in the different conditions (text vs. picture & text) do not disagree on whether Gerald knows he has seen a real house; 65.8% of all respondents attribute knowledge with no significant difference between the two groups (χ² (1) = 0.38, p = 0.538). So, it does not appear that the reason people do not attribute knowledge to the protagonist of the fake-barn scenario, is not because they struggle with understanding the scenario (in the sense of ‘filling in the details’). However, as we have discussed in Bergenholtz et al. (Forthcoming), there is a sense in which adding a pictorial representation improves participants’ comprehension of the scenario.

As part of our study, we asked three comprehension questions in total.

1. (1) Are there two people in the car? (0 exclusions).

2. (2) Does Gerald think he saw a real house? (8 exclusions).

3. (3) Did Gerald see a real house? (47 additional exclusions).

Presenting the text-based thought experiment together with an illustration of the situation described in the text-based version led to a significant and drastic reduction in the number of people who failed the comprehension questions (Bergenholtz et al. Forthcoming). In the text-only scenario (N = 82), 75.6% answered yes to the final comprehension question, while 91.4% of the subjects (N = 313) who encountered a picture in addition to the text answered yes (leading to a sample of 348 who passed all comprehension tests). This stark difference is statistically significant (χ² (1) = 15.403, p < 0.0001). Notice that the failure rate of the text-only scenario is very similar to Colaço et al. (2014), who excluded 27.7%.

Yet, while the conditions do not lead to different overall conclusions about the fake-barn scenario, we cannot, in principle, rule out that the sample reduction implied by the text scenario could be obscuring an unidentified variable influencing answers to the question about whether Gerald knows that he saw a real house (Bergenholtz et al. Forthcoming). Furthermore, we want to emphasize that it is unlikely that only those who now pass the comprehension tests are being influenced. We speculate that it is more likely that the average understanding of those who otherwise would have passed the comprehension test anyway, is also enhanced.

4.1. Replication of Colaço et al. (2014): demographic variables

As mentioned earlier, it was suggested by Colaço et al. (2014) that people of different ages vary in their fake-barn intuitions, and they offered three explanatory hypotheses for this variance. We aimed to replicate their findings and test the suggested explanatory features of risk aversion, apparency of knowledge due to previous experience (defeasibility of knowledge) and epistemic subjectivism. We also capture further demographic variables (gender and education) in order to ensure that any age effect might not be an artefact of an unbalanced sample.

In contrast to Colaço et al. (2014) we find no relationship between subjects’ knowledge attribution and their age. In a logistic regression including age and gender, only gender emerges as a significant variable (OR 0.572, p = 0.017, N = 348), which we expand upon below. In fact, if we use the median age (35) as a cut-off point, the average response to the relevant question (does Gerald know) is identical, since 65.8% of both young and old individuals attribute knowledge. One might speculate that this cut-off point is too low, and only the oldest participants respond differently. However, no significant difference emerges when comparing individuals aged 50 or older vs. those under 50 (χ² = 0.541, p = 0.462).⁸ Seventy-three of our respondents are 50 or older, which means the sample size allows more confidence compared with previous studies.⁹ Now, all our respondents are US nationals and MTurk users, and while these MTurkers have been documented to be reliable and representative (Hauser and Schwarz 2016) we don't know if respondents from other cultural backgrounds would lead to an age-related variance.¹⁰

In terms of gender, 72.2% of women (N = 169) attribute knowledge to Gerald, while only 59.8% of the men (N = 179) provide such a response, a difference that is statistically significant (χ² (1) = 5.95, p = 0.015). Given the sample size we feel reasonably confident about the differences in this particular MTurk sample, while any generalization to other contexts of course is more uncertain.

4.2. Risk

Citing results indicating a relationship between risk attitudes and age (Bonsang and Dohmen 2012), Colaço et al. (2014) suggested that risk aversion might have played a role in the age-variable result they obtained. They speculated that “more risk averse people may judge that more is at stake in a fake-barn case than less risk averse people” (Colaço et al. 2014: 207). And that if “knowledge-attribution is stake-sensitive, then older people should be less likely to attribute knowledge in fake-barn cases” (2014: 207). Independently of the agenda tied to explaining why age might be relevant to knowledge attribution relative to stakes, the question of whether risk attitude is relevant for knowledge attribution is quite interesting. It would seem intuitively plausible that something like risk attitude could be tied to knowledge attribution, and especially in a context where knowledge is a matter of luck, as is the case in a fake-barn scenario, risk attitude should be correlated with judgements about whether the protagonist knows or not. We therefore included a risk-profiling question (following Dohmen et al. 2011):

Risk Question: “How do you see yourself: are you generally a person who is fully prepared to take risks or do you try to avoid taking risks?” Participants were asked to respond on a 11-point Likert scale, 0 (“Not at all willing to take risks”) – 10 (“Very willing to take risks”).

In line with Dohmen et al. (2011), our results show that women score significantly lower on average (mean 4.41, SD 2.55) compared with men (5.61, SD 2.43) on this risk measure (t(346) = 4.50. p < 0.0001). When using ‘Gerald knows’ as a dependent variable and risk as an independent variable in a logistic regression, risk is not a significant explanatory factor (OR 1.044, p = 0.325, N = 348). Even when binning the risk data into three different variables, no significant relationship emerges.¹¹ Risk is also not a significant explanatory factor for explaining if Gerald knows, when analysing women (OR 1.04, p = 0.52, N = 169) or men (OR 0.99, p = 0.99, N = 179) in isolation, nor is there an interaction effect of gender and risk (OR 1.04, p = 0.63, N = 348). We can thus show that risk – at least when relying on this particular and widely used measure – is not a significant explanatory factor in this fake-barn scenario, and our (and Colaço et al.'s 2014) initial intuition (that risk aversion is negatively correlated with knowledge attribution) does not find any support. However, it is striking that women, who are found to be more risk averse (by Dohmen et al. 2011), are more likely to attribute knowledge.

4.3. Defeasibility of knowledge

As a second (and separate) proposed explanatory hypothesis, Colaço et al. (2014) hypothesize whether previous experience with “apparent knowledge” (beliefs initially appearing to be knowledge but that turn out not to be) may have influenced the result obtained – i.e. the result that older people are more likely to withhold knowledge attribution in a fake-barn case. They speculate that the older you are, the “more experiences of merely apparent knowledge” (2014: 207) you may have had. And the more such experiences you may have accumulated, the more likely you may be to take such considerations into account when contemplating situations in which knowledge might turn out to be only apparent.

As we understand Colaço et al.'s (2014) suggestion, what they are hypothesizing is that the subjects’ conception of knowledge (and so that of people in general) will evolve over time, since the older you are, the more false knowledge ascriptions you have witnessed. More specifically, they suggest that the older you are, the more likely you are to embrace a conception of knowledge, according to which it is understood that knowledge attributions are hedged in the sense that ‘knowledge claims’ may be overturned in time (and so people's in general) – that defeasibility of knowledge is correct: that you can know p at t₁ without knowing p at t₂ (and not for the reason that you stop believing p, but because you acquire defeating evidence). During our lives, we learn that what we thought were instances of knowledge, turn out not to be instances of knowledge. Based on this hypothesis, we included a question in our study aiming at clarifying participants’ underlying knowledge conception, with a specific focus on apparent knowledge.

We asked our respondents: “In your opinion – is it possible to know something that may turn out to be false?” (1–7 Likert scale, 1 = never and 7 = always), obtaining an average response of 4.56 (SD = 1.24, N = 225). A logistic regression shows that participants’ responses to this question are not significantly related to the question about whether Gerald knows (OR 0.98, p = 0.84). Responses to this question are also unrelated to age, gender and our risk measure. This shows that respondents who – in response to a general question – acknowledge that one can have apparent knowledge, are not more likely to attribute knowledge to Gerald in this particular scenario. Therefore, based on our approach, we find no support for the hypothesis proposed by Colaço et al. (2014). There is no significant difference between how men (average answer of 4.44) and women (average answer of 4.68) answer the question (t-test, df = 223, p = 0.151).

Yet, we find it noteworthy that only 11.6% say that it is rare (or less frequent) that something one knows can turn out to be false, while the vast majority of respondents state that this happens at least occasionally (mode response), if not frequently (see Figure 2). In other words, knowledge is considered defeasible in the sense that it may be overturned. We will get back to this point in the discussion.

Figure 2. Number of responses in percentage to each category for the question “In your opinion – is it possible to know something that may turn out to be false?” N = 225.

4.4 Epistemic subjectivism

As a third proposed explanatory hypothesis, Colaço et al. (2014: 207) hypothesize whether “younger people may be more likely to tacitly endorse some form of epistemic subjectivism than older people”. They themselves consider this option the least plausible. Colaço et al. (2014: 207) understand subjectivism as the view that: “if it seems to an agent that she knows that p, then she knows that p”. We therefore asked our participants (a) if it seems to Gerald that he saw more than one real house, and (b) if Gerald knows that he saw more than one real house. Both questions allowed a yes or no answer. Eighty per cent of our respondents state that it seems to Gerald he saw more than one real house. Comparing participants’ responses to the question about whether Gerald knows he saw a house reveals a significant correlation, i.e. if one states Gerald knows he saw more than one real house, one is more likely to attribute knowledge to Gerald (χ² (1) = 17.457, p < 0.0001), and 29.3% say yes to the second question, claiming that Gerald knows he saw more than one real house (N = 225).¹² In the following we focus only on those respondents that answered yes to the core question (does Gerald know he saw a house, N = 145), in order to investigate how they assessed all the houses, not just the focal, real house – 71.7% say yes to the first (seem) question, while 41.4% say yes to the second (know) question. We think the key data point is that out of the 104 participants that said yes to Gerald knowing he saw a house, and that it seemed to Gerald he saw more than one real house, 51.9% state that Gerald knows he saw more than one real house.

We were quite surprised to find that so many participants would be of the opinion that the protagonist knows that there is more than one house in the area. As we have made clear above, it is not a matter of participants not comprehending the scenario at hand, since they pass the comprehension questions (especially when offered the pictorial representation). This seems to indicate that people take the fact that ‘it seems to Gerald that he sees’, to be sufficient for Gerald to know. But, if this really is the case, then there is reason to think that a high proportion of participants are, in the words of Colaço et al. (2014), subjectivists.

5. Discussion

With our study, we have tried to advance the debate on (1) intuition variance and (2) studies aimed at uncovering how people in general think about knowledge. Colaço et al. (2014) argue that age is explanatorily relevant when it comes to knowledge attribution in fake-barn scenarios. Although we did not manage to replicate their study and thus did not find evidence to support the claim that age is explanatorily relevant in this respect, we believe their suggested strategy of explaining factor(s) underlying intuition variance to be correct – and we have attempted to do so empirically. Before we dive into these possible underlying mechanisms, we also note the gender differences in knowledge attribution that our sample reveals. The recent debate between Knobe (2019a, 2019b) and Machery and Stich (2019) on the extent of intuition variance across demographic factors doesn't focus on gender as a potential demographic divide, probably because relatively few studies have investigated this issue. Buckwalter and Stich (2014) provided evidence suggesting that women have different philosophical intuitions from men across a variety of philosophical thought experiments. However, Adleberg et al. (2015) failed to replicate this. Nevertheless, they find that 57.1% of women vs. 44.2% of men attributed knowledge in a Gettier case, a descriptive difference that is not statistically significant given their total sample size of 136. With a risk of over-interpreting the numbers, we note that 72.2% of women and 59.8% of men attribute knowledge in our sample of 325, a percentage difference very similar to Adleberg et al.'s (about 12.5%).

These results do seem to suggest that there might exist a gender-based intuition variance with respect to knowledge ascription. As we have suggested, noticing that this is the case is of course interesting in itself, but it would be of even more interest to be able to pinpoint what may underlie such a potential gender difference. One possible avenue of explanation explored by us was to correlate gender-based intuition variance to gender-based risk profile differences between men and women. As it is often suggested that women are more risk averse than men, we should expect to see this reflected in our measure of risk. Furthermore, if women are more risk averse than men, we should expect to see fewer women attributing knowledge to Gerald. However, we found that although women are more risk averse than men (as expected by the literature, Dohmen et al. 2011), they are also more willing than men to attribute knowledge. And so, it appears that risk is not explanatorily relevant in explaining gender-based intuition variance in knowledge ascriptions. We did not attempt further potential explanations underlying gender-based intuition variance.

More generally, our results show that the first hypothesis of Colaço et al. (2014) concerning risk is not supported. The assumption was that more risk-prone individuals would be more likely to attribute knowledge in a barn case. The risk measure developed by economists (Dohmen et al. 2011) has been validated by comparing the self-perceived response to behaviour in lotteries, documenting that the question should be able to capture how an individual would behave in a situation with real stakes. One could speculate that a risk measure developed by economists in order to capture the impact of monetary stakes does not match the kind of uncertainty involved in a Gettier scenario. If that is the case, an alternative risk measure has to be applied.

Another of our aims was to investigate people's knowledge conception and try to answer the question: do people in general adhere to a similar concept of knowledge to philosophers? The question is posed on the assumption that people may possess a conception of knowledge according to which what counts as knowledge may change over time. Thus, we asked: “In your opinion – is it possible to know something that may turn out to be false?” Our results show that many people do in fact believe this to be the case, which supports the claim that people hold a defeasibility of knowledge view. There are, however, different ways of questioning our method here, due to the vague nature of the speculations under investigation.¹³

The question that we pose is debatably ambiguous between different readings. One reading challenges the standard factive view of knowledge. Ordinarily, knowledge is taken to be factive in the sense that if a person knows that p, it follows that p. So, straightforwardly, if someone knows that p at some time t1, then it will be true that p at t1. So, in order to deny this, one would have to deny that knowing p (at t1), entails p (at t1). We doubt that this is the most obvious way of understanding the question we posed.

Another way that the question may be ambiguous is due to the room for interpretation that it leaves for participants. Maybe participants interpret the question: “In your opinion – is it possible to know something that may turn out to be false?” in the following way: ‘Imagine some claim that is currently unknown – it may turn out to be true, it may turn out to be false. Now, do you think it is possible to come to know whether this claim is true or false?’. If that is roughly how participants read the question, then our results would not indicate that the participants hold a defeasibility of knowledge view in the way that we suggest – so there is an open question as to whether the question that we ask indeed measures something like defeasibility, or if it measures belief in the possibility of epistemic progress.

As we have pointed out, according to our results most people believe that the protagonist knows, even if it happens to be by an instance of luck. But even more surprisingly, we found that people do not reserve their attribution of knowledge to cases of ‘lucky’ knowledge. When asked whether Gerald knows that there is more than one house, a large proportion of people said that he did.

When we couple this with our responses to the question about knowledge:

Q56: Does it seem to Gerald that he saw more than one real house?

Q57: Does Gerald know that he saw more than one real house?

We find that 51.9% of all the participants who claim Gerald knows answer affirmatively to both these questions. This gives room for different interpretations. One interpretation is: it would appear that a large portion of participants are non-factualists about knowledge, i.e. they are not merely inclined towards the view that knowledge is defeasible in the sense that they recognize that we can be wrong in having believed something to be an instance of knowledge. They are of the opinion that someone can know something, regardless of whether it is the case – that one's experienced state of justification is simply sufficient for knowledge. So, agreeing that it seems to Gerald that he has seen more than one real house, leads 51.9% people to say that he knows it. This, we suggest, supports the idea that many people are in fact epistemic subjectivists. However, we cannot straightforwardly infer that these people believe that ‘seeming’ is sufficient for knowing, because there could be different alternative explanations for our result beyond those suggested by Colaço et al. (2014).

As we are also interested in intuition variance between trained philosophers and lay people with no philosophical training, we should consider to what extent we can expect the fake-barn case to evoke uniform intuitions among philosophers. Most philosophers agree that general Gettier cases evoke the intuition that the protagonist does not know, but perhaps a number of philosophers think differently about the fake-barn case.¹⁴ The fake-barn case, of course, only works as a counterexample to the justified true belief account of knowledge in so far as it elicits the intuition that the protagonist doesn't know, and we take the rich discussion of the thought experiment to be evidence that a significant number of philosophers share this intuition. This does not mean that some philosophers do not disagree. Lycan (2006) for example has suggested a view that implies that the protagonist in the fake-barn case actually knows that there is a barn. Lycan however, also recognizes, that since he is rejecting the widespread intuition that the protagonist does not know, the onus is on him to explain it away. As Colaço et al. (2014: 199) observe “many philosophers have the intuition that the protagonist's belief is not an instance of knowledge”, and we do not wish to suggest anything stronger than this. This means that our results regarding the expertise defence can be considered relevant only to those among philosophers who share the intuition that the protagonist does not know, and who therefore find that there is a relevant difference to be explained between philosophers who hold that intuition and people with no philosophical training who don't.

A different kind of general worry one could have concerns our experimental setup, and that is that it is not entirely clear how people see the scenario presented. Perhaps they take the protagonist to be blameless for his/her belief formed under the conditions of the thought experiment and answer questions based on that impression. The reasoning might go like this: The protagonist can in no way be blamed for holding their belief that there is a house (as each and every one of us would be likely to form beliefs of an identical kind) in a fake-barn country scenario. So, when a protagonist like Gerald is taken to be blameless for his beliefs, he is ipso facto also taken to be justified. So perhaps participants reason like this: Gerald has the impression of seeing more than one house. When asked whether Gerald knows that he has seen more than one real house, rather than taking the third-person perspective, people adopt the point of view of Gerald, and argue from his perspective that he knows (based on it appearing to him that he knows). What people may be taking it upon themselves to respond to here is the question of whether Gerald has good justification for believing that there is more than one real house, and equating this with knowledge. Now this ‘knowledge’ that there is more than one real house, may turn out to be false (as it actually is), but Gerald (from his own perspective), according to this line of reasoning, still knows that he saw more than one real house. Participants might not distinguish between having the visual impression as of seeing a house, from having a visual impression generated by a real house. If participants take the stand of the protagonist and adopt the view that, since the protagonist is blameless in knowing in the case of the real house, then he is equally blameless when it comes to knowing in the cases of the fake houses. So rather than denying that he knows in the case of the real house, the participants take the concept of knowledge to apply in all the cases indistinguishable (from the protagonist's point of view) from the one where he has knowledge (in their view) – despite being presented with obvious examples of fake houses, to prove the contrary.

One might worry then that our design does not allow us to disambiguate between epistemic subjectivism and mere protagonist projection, where participants interpret knowledge questions as questions about ‘what would you think if you were in the protagonist's shoes?’. Although pictorial representation might play a role in helping people understand the scenario, we do not think that it would help block protagonist projection. Therefore, it is clear that further studies need to be conducted in order to really substantiate the claim that we find evidence for epistemic subjectivism. For that reason, we also have to hedge our conclusions. We cannot confidently infer that a large proportion of people are subjectivists. But in our experiment, where we suggest three different explanatory hypotheses for why people attribute knowledge in a fake-barn scenario, we can confidently infer that we cannot reject the hypothesis that a large proportion of people are subjectivists about knowledge (under Colaço et al.'s (2014) definition) to some degree. We can, however, say that we find no support for the other suggested hypotheses.

There is, of course, a further question that can be raised about whether the suggested definition of epistemic subjectivism is the standard definition of subjectivism regarding knowledge, but we won't go into that any further here. We simply follow Colaço et al. (2014), who suggest the following definition of subjectivism: “if it seems to an agent that she knows that p, then she knows that p”.

Now, whether we take our results to support the claim that a large proportion of participants are subjectivists about knowledge, or whether something different may be going on along the lines of what we just suggested, we believe that our results tell us something interesting about the fake-barn case, and potentially about thought-experiment based experimental philosophy in a broader sense.

When it comes to those among the general public who hold a ‘justified true belief’ account of knowledge, it might be the case that they share intuitions with those philosophers who have responded to the fake-barn scenario by denying that Gerald knows, because of the luck involved in the barn case. But what we find is that many people may, from the outset, have a different concept of knowledge (from philosophers). Now, if this is the case, we cannot simply run standard philosophical experiments on the assumption that people share what we take to be standard philosophical points of view or concepts as their point of entry. A case like the fake-barn scenario is not designed to map this (as it captures intuition variance but gives us no explanation of variations), and different kinds of studies need to be devised in order to get an idea about whether it is the case or not. In our study, we believe that we have collected data to suggest that this is worth further study. Finally, we want to close our discussion by acknowledging that the inferences are based on an Mturk sample. While such samples are generally established to be reliable and useful if carefully executed (Thomas and Clifford 2017; Hauser et al. 2019), responses cannot be assumed to represent all populations (Henrich 2020). We hope future research will replicate our particular questions, as well as investigating the robustness of our operationalization of the hypothesis by trying out additional approaches and variables.¹⁵