We conducted a web-based study investigating whether the probability of deriving four types of pragmatic inferences depends on the degree to which one has traits associated with the autism spectrum, as measured by the autism spectrum quotient test (Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, 2001). In line with previous research, we show that, independently of their autism spectrum quotient, participants are likely to derive those pragmatic inferences that can be derived by reasoning solely about alternatives that the speaker could have used. However, if the derivation of the pragmatic inference draws upon more complex counterfactual reasoning about what the speaker could have said, the probability that it is derived decreases significantly with one’s autism quotient. We discuss the consequences for theories of pragmatics in autism and for linguistic theorizing in general.