Second language learners’ reading is less efficient and more effortful than native reading. However, the source of their difficulty is unclear; L2 readers might struggle with reading in a different orthography, or they might have difficulty with later stages of linguistic interpretation of the input, or both. The present study explored the source of L2 reading difficulty by analyzing the distribution of fixation durations in reading. In three studies, we observed that L2 readers experience an increase in Mu, which we interpret as indicating early orthographic processing difficulty, when the L2 has a significantly different writing system than the L1 (e.g., Chinese and English) but not when the writing systems were similar (e.g., Portuguese and English). L2 readers also experienced an increase in Tau, indicating later-arising processing difficulty which likely reflects later-stage linguistic processes, when they read for comprehension. L2 readers of Chinese also experienced an additional increase in Tau.